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The Florida Merit Schools Program is described and 
evaluated in this report. Program goals were to increase stuot-nt 
academic performance and to provide economic incentives to school 
staffs. Research phases involved: dociuaent analysis of school 
district plans submitted to the state Department of Education; 
interviews with school personnel from five districts; mailed 
questionnaires to teachers, principals, and other administrators; and 
a stakeholders' conference. The results are presented in two 
sections: a descriptive and impact analysis. Findings indicate that 
the program is generally viewed as having a positive impact on 
instructional practices. Its effect is greatest in districts that 
award merit school dollars to purchase materials and equipment only, 
cind the least effect in those that award salary supplements. The 
relationship between merit school winners and improved student 
achievement is unknown, due to the use of standardized achievement 
test scores. Recommendations are made for increased school autonomy 
in determining optional criteria and for the distribution of award 
money in forms other than salary supplements. Appendices include the 
interview guide, mailed questionnaire, conference program and 
participarits, correspondence, examples of statistical models, survey 
result attitude index analysis, and socioeconomic school data. 
(LKI) 
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EXECtJTIVE SUMMARY 



I. RFP Research Questions: 

a. In what ways is the program acting as an incentive to 
improve instructional practices? 

The program is generally viewed as having a positive impact on 
the improvement of instructional practices. This impact is 
greatest in those districts that award merit school dollars to 
purchase materials and equipment only., and least in those districts 
(with the exception of Dade County) that award salary supplements, 
(pp 57-59) 

Is there a relationship between improved student achievement 
and Merit School winners? 

There is probably no relationship, but this question can not 
be clearly answered. The difficult lies in the definition of 
student achievement. The program defines it in terms of 
standardized achievement test scores, interpreted according to a 
statistical model devised by each district. Since it is impossible 
to ascertain whether, student achievement is improving by examining 
the results of the standardized test scores, this question can not 
be answered, {pp. 51-57) 

b. What types of schools win? Is there a disproportionate 
number of upper socioeconomic level schools which have received the 
awards over the years? Are there other school characteristics 
common to Merit School winners? 

No, a disproportionate number of upper socioeconomic level 
schools have not won merit school status. In those districts in 
which a high percentage of schools receive merit school status each 
year^ socioeconomic status has no bearing on the program. In some 
larger districts there is a very weak relationship between SES^ as 
measured by free and reduced lunch data, and merit schools. In 
terms of other characteristics, the percentage of minority students 
has only a weak relationship with merit school status in some 
districts. (pp. 48-51) 

c. What has been the impact of this program on personnel 
practices? Has there been an incentive as a result of funding for 
teachers to transfer to Merit Schools, thus depleting the ranks of 
better teachers at other schools? 

There has been very little impact of the program on personnel 
practices such as teacher transfers. Only in Dade County might 
there be a weak relationship between teacher transfer and merit 
school status. (pp.65) 
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d* How do schools spend the award money? What proportion goes 
to teachers, administrators and support staff/ and what part goes 
into instructional materials and other areas? 

Approximately 17 of the participating districts in 1987--88 
spent most of the award money in the form of salary supplements, 
while the other 12 distributed most of it to purchase materials and 
equipment. These figures are only approximate, as several award 
patterns were discovered. The distribution of award money proved 
to be a key factor in explaining the impact of the program* 
(pp 30-33) . 

e. How do the various district plans for Merit Schools differ 
and how are they similar? 

In terms of the statistical models chosen by the 29 
participating districts in 1987-88, all models were unique. Very 
few districts adopted any of the suggested models found in the 
annual memorandxim from DOE announcing the program. In terms of the 
optional criteria, 18 districts adopted criteria that were fixed by 
the district; 2 districts adopted competitive criteria, while 9 
districts allowed some degree of discretion by the individual 
school building in the choice of criteria. {pp 12-27) 

f . What percentage of the schools in participating districts 
have received Merit School dollars? 

Over the five years of the program, the percentage has varied 
from 0 to 100%. In the 1987-88 year, 9 districts awarded from 71- 
100% of their schools as merit schools; 10 districts awarded 36- 
70%; and 9 districts awarded from 14 to 36%. (pp. 27-33) 

g. How have criteria established for Merit schools changed 
over the years? 

In most districts, the criteria have changed very little over 
time. One reason why this is true is that participation over time 
has been mixed, as only 15 districts participated in all 4 years of 
the program (1984-88); 4 have participated in 3 years; 13 have 
participated in only 2 years, and 10 participated only 1 year. 
Another indicator is that percentage of merit schools awarded has 
remained consistent over time for roost districts. (pp 27-33) 

h. How are Merit Schools measuring and documenting the 
achievement of their goals? 

For the standardized achievement test score part of the 
program, the scores are determined by the district office and 
reported to tne schools. For the optional criteria, individual 
schools are reporting achievement of these goals to the district. 
Schools are required to maintain documentation, but districts 
rarely if ever ask schools to produce this documentation, (p. 33) 



II. other Relevant Impacts of the Merit School Program: 



!• Perception of the program does not differ significantly 
according to school level (elementary, middle/ junior high school, 
high school) • Perceptions do differ significantly across three 
categories of districts differentiated by the primary method of 
awarding merit money: 1) salary supplements; 2) instructional 
materials and equipment; and 3) Dade county (salary supplement) • 
(pp. 41-45) 

2. The Overall impact of the program is generally viewed as 
positive, with school administrators more positive than teachers, 
and educators from materials only districts more positive than 
those from salary supplement districts, (pp. 45-48) 

3. A majority of teachers and administrators in the materials 
only districts and in Dade agreed that the program has had a 
positive impact on instructional practices, while the salary 
supplement schools are much less in agreement, (pp. 57-59) 

4. The program is perceived to have a very positive impact on 
school based management practices in all districts, (pp. 59-62) 

5. In terms of providing an adequate economic incentive for 
teachers, the program is perceived negatively in salary supplement 
districts, where for the most part the amount of salary supplement 
is low. The program is providing a positive economic incentive in 
the materials only districts, (pp. 62-63) 

6. The major deficiency of the program has been the lack of a 
formal opportunity for districts to share and discuss strategies, 
problems and successes, (pp. 65-68) 
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ASSESSMENT OF THE MERIT SCHOOLS PROGRAM: 
PROJECT ACTIVITIES— FINAL REPORT 

The goals of the Florida District Quality Incentives Program, 
better known as the Merit Schools Program, are (1) to increase the 
performance of public: school students; and (2) to provide economic 
incentives to instructional and other authorized personnel. In 
order to more specifically address these goals, the STAR grant RFP 
identified several specific questions. These questions can be 
categorized into two groups: 1) the nature of the processes 
associated with the program and program characteristics; and 2) the 
impacts of the progrcim. 

This first section of the final report provides a description 
of the activities undertaken as part of this STAR Grant project, 
with an explanation for any activities that represent a change from 
the original proposal. Overall, the schedule identified in our 
response to the RFP has been met. 

Phase One: Research Refinement July - October, 1989 

The major task of phase one was an intensive review of data 
found in the records collected by the ?lorida Department of 
Education. These records consisted of 1) the plans submitted by 
each district for the year 1988-89, and 2) the year end fiscal 
reports submitted by allL districts for the entire five year period 
of the program. We met with DOE personnel on July 25, speaking 



2 

with Dorothy Routh, John Winn, Brian Curry, Janice Smith-Dann, 
Rufus Ellis and Janine Blomberg* The major refinements to our 
original proposal that resulted from our discussions included the 
following. First, we agreed that data would not be obtained from 
districts that chose not to participate in the program. 

We visited Senator Jack Gordon in Miami Beach on August 16 to 
obtain his views on the merit school program. As the author of the 
merit school legislation, his comments were very helpful in gaining 
additional insights into the goals and objectives of the program. 

Second, it became apparent from discussions, and confirmed 
from subsequent analysis of district plans, that a review of plans 
for all districts for all five years would not be preferable. 
Instead, it was decided that an intensive review of the plans for 
the yecir 1988-89 would furnish a view of the full range of plan 
content possible under the programmatic guidelines. Plan content 
for any given district has n. t varied much from year to year. In 
addition, only 14 districts have participated for all five years. 
Some initial assessment of plans was made in DOE offices on July 
2 5-26 , Copies of all plans for 1988-89 weire furnished by DOE and 
a much more intensive analysis was performed in the weeks 
following. 

We first attempted to categorize plan content for both the 
standardized test (with accompanying statistical model) criteria 
and the optional criteria. Each of us read all 29 plans in great 
detail and first attempted to use a five category breakdown to 
describe the statistical models used by the districts. It was 
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finally decided that these five categories were not useful, as each 
of the 29 districts employs a different statistical model. With 
the optional criteria, we more successfully grouped district plans 
into four categories, entitled competitive, fixed by district, 
variable by school (minimal) , and variable by school (more 
substantial) . Analysis of both the statistical models and the 
optional criteria is reported in the next section of this report 
(pages 11-27) . 

The second major set of records reviewed were the year end 
fiscal reports. These were obtained for all four years of the 
program, and furnished useful descriptive data, including the 
number and percentage of schools identified as meritorious, and the 
categories (and percentages of dollars) that identified how each 
district spent its merit school dollars. Analysis can be found in 
p\j. 28-34, tables 5-8. 

In preparation for phase two, and in consultation with DOE, a 
letter bearing Commissioner Castor's signature was sent to all 
participating districts on September 20, 1989, requesting that a 
list of merit schools for the past four years of the program be 
sent to us. This information was needed so that we could plan to 
interview school personnel from schools that had achieved merit 
school status as well as those that had not. By January, 1990, 
approximately half of the districts had responded. Reminder 
letters to the other districts were sent on January 8, 1990. By 
March, approximately 90% of the districts had responded. 

In an attempt to elicit feedback from key administrators. 
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principals and union representatives, we decided to take advantage 
of the annual meeting of the Florida Education Research 
Association, held in Tallahassee on November IS-'IS, 1989. We 
requested and received approval to hold a seminar concerning the 
merit schools program. To help generate discussion, we coauthored 
a paper entitled ''The Florida Merit Schools Program: an Initial 
Assessment". The following people participated in the seminar: 
Yvonne Burkholtz and Robert Sipes, Dade County School District; 
William Piotrowski, Leon County School District; Micheal O'Farrell, 
Senate Education Committee; Richard Layer, FEA; and Molly Read, 
FTP-NEA- 

Phase Two: School Interviews — September to December, 1989 

The primary focus of phase two was the in person interviewing 
of selected personnel from a sample of school districts. The 
purpose of the interviews was to elicit additional insights to more 
fully prepare for the mail questionnaire that constitutes phase 
three. Using the categories reflected in Table 8, p. 33, we 
selected five districts in which to perform interviews. These are: 
Dade, Brevard, Seminole, Marion, and Pinellas. Prior to our 
visits, we contacted the appropriate district program coordinator. 
In most cases, this person helped us schedule interviews in the 
schools. Interviews were held in the following districts on the 
following dates: 

r 12 



Dade 



September 25-26 




Marion 



Brevard 



Seminole 



Pinellas 



November 16-17 



Noveiaber 20*^21 



November 20-21 



December 11-12 



I 

i 



A set of open ended questions was developed (see appendix A) • 
The questions were chosen as a result of the initial discussions 
with the DOE personnel, Senator Gordon, and a review of related 



literature. Using these questions, we attempted to ascertain a 
range of attitudes toward the program as well as better understand 
the relevant activities and processes. 

During this phase we also obtained from DOE data concerning 
the socio-economic status (percent free and reduced lunch) and 
ehtnic group memebership of students in each Florida public school. 
Analysis of this data, performed in Phase IV, helped us assess the 
extent to which the merit schools program was free of bias related 
to the socio-economic status and minority status of students. 

Phase three: Mail Survey January — March, 1990 

During January and February, 1990, we drafted, revised, and 
pretested four survey instruments. Two of these were sent to 
teachers, one to principals and other school administrators, and 
onu to district administrators. To permit comparisons between 
respondent groups, a set of common items were included in the 
surveys. Copies of the instr^J^ents can be found in Appendix B . 



Almost all of the i'.ems were close-ended* The survey asked 
respondents to indicate the extent: to which th ^ agreed or 
disagreed with a series of statements. A scale of 1 to 5 was used, 
with 5 strongly agree, 1 strongly disagree, and 3 undecided. The 
survey sent to one group of teachers included statements concerning 
topics such as 1) overall impressions of the merit school program; 
2) L yact of the program on student learning; and 3) impact of the 
program on school based management practices. A supplemental set 
of statements concerning the effect of competition fostered by the 
Merit Schools Program was also sent to this first group of teachers 
(see appendix B) . A second group of teachers were surveyed in 
terms of overall impressions plus their feelings regarding the 
standardized testing aspect of the program. A survey sent to 
principals and school administrators included the same set of 
statements found in the survey sent to the first group of teachers. 
Finally, district administrators were asked to respond to a smaller 
number of separately composed statements. 

Surveys were mailed on February 16, 1990. Teachers and school 
administrators were sampled according to a sampling plan that is 
described on pages 37-40 of this report. Of the 3067 teachers 
surveyed, returns numbered 1138 or 37%. For principals and school 
administrators, 387 were surveyed, resulting in 210 returns, a rate 
of 54%. Tenty nine district administrators whose district 
participated in the program were surveyed, with 26 (90%) returning 
the surveys. 

Given the reasonably high rate of return, it was decided that 



no follow up telephone calls nor additional mailed surveys were 
necessary. Data was entered into the computer from retximed 
surveys until April 6, 1990* This date was chosen to allow 
sufficient time to prepare analysis for those who agreed to 
participate in the Stakeholders conference held at the University 
of Central Florida on May 10-11, 1990. 

Phase Four: Stakeholders conference, Preliminary final report. 
Final report— March , 1990 to June 30, 1990 
The purpose of the conference was to review the survey 
results, discuss reasons for success or failure, and to identify 
possible recommendations for future improvements in the program. 
A list of issues and questions were distributed to each participant 
ahead of time (see Appendix C) , along with summaries of initial 
data analysis. Twenty four people attended the conference, most of 
whom were school administrators. A list of attendees is found in 
Appendix D. 

As a result of the conference, and at the request of 
Department of Education officials, a preliminary final report was 
drafted and distributed to key education officials and legislators. 
This report, mailed on May 18, 1990, commented on proposed 
legislation and provided some initial data analysis. The proposed 
optional Phase Five, in which we would work with legislators to 
write recommended changes in the Merit Schools program, did not 
occur as originally proposed. However, at the request of DOE 
officials, we provided our analysis of the recommendations found in 
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the report entitled "Florida's Quality Instruction Incentives 
Program: Allocation Patterns of Merit Schools - 1984-89" . This 
analysis, submitted on April 12, 1990, can be found in Appersdix E. 

The remaining time for the grant period has been spent 
perfonning further data analysis and preparing the final report. 



THE FLORIDA MERIT SCHOOLS PROGRAM: A DESCRIPTIVE i\NALYSIS 



INTRODUCTION 

The Florida Quality Instruction Incentives Program, comiaonly 
known as the Merit Schools Program, has been funded for five years 
and currently distributes ten million dollars annually among 
participating school districts. In the 1988-89 academic year, 29 
of the 67 county districts participated in the program. During its 
five year history, there has been no review or assessment of how 
the program is working or how the funds are spent by individual 
districts. 

Data for this descriptive analysis was gleaned from 
interviews with selected district administrators and teachers, plus 
several sets of documents furnished by the Florida Department of 
Education. These documents include, first, all, approved plans for 
the 1987-88 academic year. Second, end of the year tinancial 
reports (see figure one) for the first four years of the program 
furnished data concerning school participation and fund 
distribution. Third, the memorandum to all school districts that 
marks the beginning of the annual merit school program furnished 
useful data. 

The following descriptive analysis reviews the following 
issues. First, demographic data concerning district participation 
is briefly reviewed. Next, in direct response to the RFP questions 
concerning the ways in which districts plans are similar and are 
different, the two major categories of merit school criteria that 
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all schools must address are discussed* Using a content analysis 
of 1987-88 plans, coraments concerning these criteria are provided. 
Third, patterns in the percentages of schools designated as iserit 
schools within each district are identified for the first four 
years of the program. Then the distributions of merit school funds 
are examined over time, with particular analysis made of how these 
distributions are linked to merit school percentages for each 
district. 

DEMOGRAPHIC RJVIEW OF PARTICIPATION IN THE MERIT SCHOOLS PROGRAM 

Tables 1 through 3 provide an overall demographic view of 
district participation in the Florida Merit School program as of 
1987-38 • As indicated in Table 1, participation in terms of region 
is approximately the same (33-43%) statewide with the exception of 
the east central region (80%). Throughout the state, total student 
enrollment in districts that participate is approximately the same 
as in those that do not. 

When the factor of size is considered. Table 2 provides a 
grouping of districts by the categories of very small, small, 
moderately small, moderate and large. Size is measured here by 
thousands of student enrollment in the district. Table 3 combines 
all the factors considered, indicating participation by size and 
region. Because the numbers in each category are relatively small, 
no real pattern emerges from this data. 



fefeTable 1: 



Listing of School Districts, Enrollment and Partaicipation 
in the 1987-88 Florida Merit Schools Program by Region 



^ "^^T^??^ />J*{jr'': '■r^>fyti^lt"^":^^:'^^^^ 



Region 



Dist 
code 
number 



Participating 



dollars enroll. 
( in Thousands ) 



1 northwest 
panhandle 



3 Bay $252 

7 Calhoun 23 

23 Gulf 25 

37 Leon 306 

40 Madison 36 

67 Washington 35 



6 of 18 or 

33% participation 



2 


northeast 2 


Baker 


$47 




shouloer 4 


Bradford 


49 




18 


Flagler 


39 




45 


Nassau 


94 




54 


Putnam 


130 




61 


Suwannee 


58 




63 


Union 


20 




42 


Marion 


316 




8 of 19 or 








42% participation 




3 


eastcentral 5 


Brevard 


$589 




31 


Indian R. 


124 




35 


Lake 


229 




48 


Orange 


1030 




56 


St. Lucy 


212 




59 


Seminole 


508 




60 


Svunter 


57 




64 


Volusia 


504 



8 of 10 or 

80% participation 



4 westcentral 25 Hardee $47 

51 Pasco 360 

52 Pinellas 1120 
58 Sarasota 326 

4 of 13 or 

31 % participation 



5 south 



13 
43 
44 



Dade 

Martin 

Monroe 



$3243 
123 
94 



3 of 7 or 

43% participation 



Total student enrollment 
of participating dists. 
in thousands 



Not Participating ' 
Dist. enroll. ^ 
code (in thousandsO^^ 
number ■ 



22 
2 
2 

25 
3 
3 



4 
4 
3 
8 

11 
5 
2 

26 



5C 
10 
19 
89 
18 
44 
5 
42 



4 
30 
89 
26 



254 
11 
8 



815 
(49%) 



RiC 



19 



17 Escanibia 

19 Franklin 

20 Gadsden 
30 Holmes 

32 Jackson 

33 Jefferson 
39 Liberty 
46 Okaloosa 
57 Santa Rosa 
62 Taylor 

65 Wakulla 

66 Walton 



1 Alachua 

9 Citrus 

10 Clay . 

12 Coltainbia 

15 Dixie 

16 Duval 

21 Gilchrist 

24 Hamilton 

38 Levy 

34 Lafayette 

55 St. Johns 



47 Okeechobee 
49 Cjceola 



8 


Charlotte 


10 


14 


Dasoto 


4 


22 


Glades 


1 


27 


Hernando 


11 


28 


Highlands 


8 


29 


Hillsborough 


118 


36 


Lee 


38 


41 


l^smatee 


24 


53 


Polk 


61 



6 Broward 

11 Collier 

26 Hendry 

50 Palm Beach 



Total student enrollment 
of nonparticipating dists. 
in thousands 



liable 2: 



&%i g|^Hifidfi§h§§rg?iil!c?rby size category 



ife^Jhiullnds ""^^^ number. Of Florida dists- 

lo? ItSSs^ diits. size category in size category 



4 I 

5 4 



8 Very small , , 26 

6 (5 thousand or less) 



7 

3 small ^ 16 

1 (6 to 14 thousand) 

1 

2 
1 

1 Mod. small H 

2 (15 to 29 thousand) 

2 



30 3 

It 2 Moderate 

42 4 



1 (30 to 69 thousand) 



50 1 

61 1 

89 2 

90 1 

105 1 , L^^^ 



137 1 
254 1 



i (over 70 thousand) 



(8/B9) 



iTable 3= Hurler of farti=ipating^(|)^|nd HonParti=ipatii,g (SP) Districts 





V. 


small 


small 


mod. 


small 


moderate 


large 


total 


Region 








HP 


P 


NP 


P 


NP 


P 


NP 


P NP 


1 


4 


7 


0 


3 


2 


1 


0 


1 








2 


5 


4 


2 


4 


1 


2 


0 


0 


0 


1 


8 11 


3 


1 


1 


1 


1 


2 


0 


3 


0 


1 


0 


8 2 


4 


1 


2 


0 


3 


1 


1 


1 


2 


1 


1 


4 9 








? 


n 


0 


1 


0 


0 


1 


2 


3 4 




11 


15 


5 


11 


6 


5 


4 


3 
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FLORIDA DEPARTMENT OF EDUCATION ^ ^ ^ ^ 

DIVISION OF PUBLIC SCHOOLS aJLkjT.^^ 
'^WANOAL MANAGEMENT ^'STRICT QUALrTY INSTRUaiON INCENTJVE PROGRAM REPORT 



A. it86_.»87 

C tHftrktNam* 



OADE 



f. AAcWittajlwCwt S 41,632 



t. OhttktWwiibwi 13 

o. oittttctPwMh t 5>440 
9. S<u<l«ntlUco»>HtonAw«nfa» 91 ^OOQ 



9. 9^otfotffSthooh.Ulf.S32(3Xf).iF*S. { 




S<hoo(3 
IMtUkt 


Sfhoett 


School 


R«€tMiif Aw«rdt 
{SUtuPurMf} 


6i(|xifKf^ for 


7*Total 
OotUrt 
Sttptmfcd 

Othtf fhtn 
lmployt9t 


fctitfUC' 

tional 


ttruc 
tionai 




178 


136 


42 


1,966 


689 


1,673,738 






47 


29 


14 


841 


401 






(a HS^h School 


24 


16 


7 


949 


333 


829,342 






27 


15 


3 


57 


13 


30,765 






27 


9 


3 


245 


89 


163,359 




















t Itinerant 


N/A 


N/A 


N/A 


253 


9 
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SIMILARITIES AND DIFFERENCES AMONG DISTRICT PLANS: THE MAJOR 
CRITERIA USED TO DETERMINE MERIT SCHOOL STATUS 

There are two major categories of criteria used to determine 

the merit school status of an individual school* First, 

Such a school shall be selected by at least 
exceeding its statistically verifiable 
expectancy level on an approved standardiiied 
test as measured by tests of verbal and 
quantifiable achievement* (CH 231*42, 3(f)) 

Although Department of Education guidelines propose a variety of 

possible statistical models, the school district does not have to 

follow any of these suggestions, and may create its own model or 

statistical process to meet this criterion. 

Second, a standard or set of standards must be selected from 

at least one of the following types: 

1) standardized tests; 

2) participation standards; 

3 ) achievement standards ; 

4) discipline standards (CH 231.42, 3(f)) 

There are no additional specific guidelines that schools must meet 
to achieve any or all of these standards. A statistical model does 
not have to be applied in the case of the non reading/math 
standardized tests* The legislation provides examples of the 
national physical fitness test or the state student achievement 
test (SSAT). The only restraining guideline is that schools must 
provide processes for recognizing student progress. Thus there 
must be a baseline percentage or figure for each standard 
identified in each district plan, plus there must be at least a 
minimal improvement over previous year's performance indicated as 



J]RLC 



24 



12 

part of the plan as well. 

In the review of the school district plans that encompass 
these two major categories of merit school criteria, for the 
optional criteria, four nominal groupings were chosen. These are: 

1) competitive — among schools; 

2) fixed by distris::t — ail schools must adhere to these; 

3) variable by school — minimal; 

4) variable by school — more substantial. 

A more in depth discussion of each of these two major 
categories of criteria follows. Analysis of how district plans are 
similar and how they differ is provided. 

The Statistically Verifiable Standardized Test Criteria 

The July 29, 1988 memorandum to district school 
superintendents on the subject of District Quality Instructional 
Incentives Program provided several examples of statistical models 
for use in selecting meritorious schools (see appendix F) . It is 
not appropriate to discuss the wisdom or appropriateness of the 
methods illustrated in that memorandum. However, the examples do 
stimulate a number of interesting questions that need serious 
review. For example, what is really intended by Method III: Student 
improvement by within-school percentile distribution? Why have 
school districts avoided the use of a Standard Error of Estimate as 
recommended in Method I? Is the use of a Standard Error of Estimate 
really inappropriate or unrealistic? 

Some of the methods vfere discussed in a set of papers authored 
for the Florida Assessment, Testing and Evaluation Sections of the 
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Division of Public Schools. The undated collection of papers 
entitled Methods For Determining Merit Schools contains chapters 
authored by Rick Nations, R. Clifford Blair, William Meredith and 
Linda Murray, Ray Turner and Robert Stephenson, and Richard Tate, 
In the papers, serious questions about the mathematical and 
statistical adequacy of some of the models have been raised. There 
is some awareness that, with respect to the models proposed by 
school districts, the task is not so much for school districts to 
defend the appropriateness of their model but for the state to 
either accept the model or prove that it is inappropriate. Because 
the program provided no additional resources for state supervision, 
it would be nearly impossible for the state to reject most any 
proposed model because the state's burden of proof of 
inappropriateness would be too great. 

The analysis of the 1987-88 district plans made as a part of 
this study indicates that a very wide array of strategies are used 
by school districts to satisfy the standardized achievement test 
criterion. The state and school districts have assumed the 
criterion to require the use of norm referenced scores in 
describing academic achievement. Perhaps standardized criterion 
referenced tests would be just as appropriate. The term 
standardized test is used in another way when discussing optional 
criteria. 

Most districts have generated unique strategies or have 
designed unique modifications of a n»ethod illustrated in the 
state's memorandum. Few districts have attempted to directly 
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implement one of the illustrated models • 

Some of the models may result in the identif icat5,on of random 
error. It is too easy to assu::ne that any deviation from a 
predicted level is due to effects of teachers or school 
characteristics not built into the model • The assximption may not 
be valid, especially when the predictor variables used to project 
expected scores are strongly correlated with both the 
characteristics of merit schools and the dependent variable • 

The statistical models described in several school district 
plans only briefly outline a general strategy* The description 
contained in the plans does ncL always include a description of the 
mathematical model used. Based on phone and personal conversations 
with a small nonrandom sample of school district test and 
measurement staff, there is reason to be concerned about the extent 
to which these staff believe in the appropriateness of the models 
being used by their districts* In some cases the models pre nted 
in the plans may not be the models actually used when the plan is 
implemented. 

The plans differ in at least the following ways: 

— Some districts use one model for all school levels 
(elementary, middle and high school) while other 

districts vary the model by school level. 

— Some districts base their calculations on all students 
for whom test data are available and others use 

selected grade levels. 

— Some districts calculate achievement scores on a 
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Student by student basis and others calculate building 
level mean scores* 

Some districts average scores across all students or 
grade levels while others identify the percent of 
students or number of grade levels within a building 
that demonstrate improvement in achievement. 

Some school districts identify gain as any increase in 
percentile score from one year to the next while other 
districts modify the required percentile increase by 
the stanine level at which the student performed during 
the base year. 

Some districts use the previous year as the base for 
comparison while others use a running mean based on the 
school's grade level mean for three or more years. 
— Some districts use a regression model while others do 
not. 

It is not always clear whether building level percentile norms 
or student level percentile norm serve as the reference point. 
Failure to attend to the difference is similar to failing to 
distinguish between a distribution of raw scorei. and a sampling 
distribution of means. 

Another major distinction within the plans can be called a 
normative vs. ipsative distinction. In the normative based plans, 
the ifleasured amount of academic gain in standardized test scores 
for each school is used to rank the schools. Merit schools are 
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either identified solely on the basis of this ranking (provided the 
school has made at least some effort in the optional category) or 
schools are assigned points base' on their relative ranking in the 
district. The achievement test points are added to points earned 
in the optional category and merit school status is assigned to a 
fixed percent of schools with the highest number of total points. 

In the ipsative based plans, each student's or each school's 
yearly achievement score is compared to the score form the previous 
year. Students or schools are required to compete with themselves. 
The required gain is often a single percentile point. In some 
districts the number of percentile points required to demonstrate 
gain varies by the level of the students' pervious achievement 
(stanine) score. Only few of these plans consider error of 
measurement. 

Because the plans vary on so many characteristics, it is 
difficult to accurately classify the plans by a simple scheme. 
Almost every school district has a unique set of strategies for 
identifying :nerit schools. 

As mentioned previously, the extent to which test scores 
impact the selection of merit schools varies widely. In some 
di-,tricts, test scores are the primary, if not the only, criterion 
provided the school makes at least some effort on one or more of a 
set of optional criteria. In other districts, test scores count 
only as a small part of the number of points required to earn merit 
school status. The optional criteria account for the major number 
of point required to earn merit school status. 
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During a few of our intarviews, some principals reported that 
they had to demonstrate to their teachers that no added work would 
be required in order to satisfy the test criterion. The school 
would have easily satisfied the requirement for every year during 
the past several years. They convinced the teachers to vote ^or 
participation in the merit school program because it would be a 
good motivator to help improve the school in other areas that 
everyone knew needed improvement. The money given to the school 
would help get the materials needed to make the school even better. 

Appropriateness of Standardized Tests 

Another set of issues has emerged from the current national 
and local discussions of the appropriateness of standardized test 
scores for differentiating between more and less effective teachers 
and schools. Based on a small set of interviews of administrators 
and teachers, articles and papers written by Florida Teachers, and 
interview data gathered for other studies in progress, it is clear 
that many of Florida's school administrators and classroom teachers 
do not distinguish betwean criterion referenced or minimum 
competency tests and standardized, norm referenced test. 

Because of local and state mandates to ensure thrt minimum 
competency and criterion referenced tests have what James Popham 
called instructional validity, school districts and schools take 
elaborate steps to ensure that the content of the such test-s has 
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been taught • Local and state objectives strongly guide teaching to 
ensure that specific skills are mastered by students. Teachers are 
required to make sure that students study and master the specific 
objectives on which the tests are based. 

Some administrators and teachers view standardized norm 
referenced tests from the same point of view. They want to make 
sure that the tests have instructional validity and they do what 
must be done to ensure, to the best of their ability, that students 
are provided instruction on what they have learned over time to be 
the content of the test. The content of standardized tests does 
not change frequently. Some teachers remember vocabulary and items 
from the test. In an effort to do what they think is proper and 
fair to students, some administrators and teachers invaliditate the 
test by building vocabulary lists derived form what they have seen 
on the tests, and create lessons to ensure that students have been 
taught the material included on the test. In addition, some 
teachers and administrators are very much aware of student 
testwiseness skills. They use a variety of materials such as 
"scoring high" and locally created materials to make sure that 
students are familiar with test item formats used on standardized 
norm referenced tests. In some cases teachers create test items 
identical to those on standardized tests but change a word or two 
in order to avoid what they think would be teaching the test or 
cheating. 

One of several problems with using test scores to identify 
relative effectiveness of teachers and schools is that the use of 
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the practices listed above is uneven among teachers in a building 
and between schools ♦ Some schools and teachers spend a great deal 
of time on standardized test preparation, while others send very 
little or no time on such practices. The impact of such practices 
and the use of standardized tests to identify more effective 
schools have recently been widely discussed under the topic of 
high'-stakes testing. 

During interviews, some teachers and administrators reported 
that they were totally unaware of the fact that test scores were 
used as a major factor in identifying merit schools and reported 
doing little or nothing to prepare students for the norm referenced 
standardized test. Other teachers and other schools had elaborate 
plans for teaching testwiseness skills with purchased and locally 
created material. Some teachers reported being in a dilemma* They 
did not like taking large amounts of time away from the regular 
curriculum to teach testwiseness and special conte'^t known to be on 
the test, a practice they believed to be unprofessional and more 
like cheating. Yet, they knew that their school needed to be 
identified as a merit school in order to boost pride, earn the 
respect of parents, or to have the money to purchase needed science 
materials, manipulatives and other items not provided by regular 
sources. They didn't like to have to "cheat or misuse 
instructional time" in order to help the school in other ways. 

Administrators, and teachers reported that a concern about the 
pressure to score high on norm referenced and mastery or criterion 
referenced tests had gradually caused them to shift their 
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curriculum to match the content of the test. They believed the 
shifting of curriculum was partly responsible for the •'test score 
creep" often visible in gradual increases in standardized test 
scores reported by local districts. Over time^ the inflated scores 
make it nearly impossible for a school to score below their 
predicted achievement level when a simple expected level is based 
on old norms. 

Some teachers and administrators indicated that school 
districts and test companies contributed to test score creep by 
providing schools with detailed item by item printouts of error 
reports called diagnostic reports. Such information seems to 
encourage them to modify their curriculum to fit the content of the 
test. They may refer to part of the test analysis as objective 
matching and criterion referenced analysis. Teachers indicated 
that they felt forced to modify their curriculum to fit the content 
of the test. It is another indicator of the failure to distinguish 
between norm referenced and criterion referenced tests. 

Some teachers and administrators are concerned about the 
requirements for ever increasing test score that seem to be built 
into some models for identifying merit schools. If a total school 
can have an impact on student scores, it is unlikely that such an 
impact can be created in a short tiine. In schools that are truly 
producing students that achieve above their expected level, the 
expected level needs to be based on indicators other than last 
year's achievement. If last year's achievement is used, students 
who continue to demonstrate the same level of achievement above 
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what would be otherwise expected will have scor^^s that indicate no 
gain and will be ignored. Some plans based on an ipsative strategy 
failed to consider that issue. It also is a problem in some 
regression models. 

It would seem that there is much work to be done before the 
long list of statistical and measurement issues can be resolved. 
The issues need closer examination in order to assure teachers, 
administrators, parents, students, and other interested persons 
that the strategies being used to identify merit schools are 
appropriate . 

Perhaps Albert Shanker was correct when he indicated in July, 
1989, that we need to continue to identify and support the 
development of strategies for promoting programs such as Florida's 
merit schools program. He stressed the need to identify 
alternative assessment techniques. Shanker focused national 
attention on Florida's merit school program and its ability to 
promote school improvement through school based management and 
better teacher- administrator cooperation. But he seemed to be 
unaware of the extent to which we rely on standardized test scores 
when he said "Schools work hard to get their students to do well on 
tests when they know students' scores will be compared with the 
scores of their peers in other schools — look at the way schools 
struggle to raise student grades on those idiotic standardized 
tests that are out there now.*' 
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Given that the law provides an extreme amount of flexibility 
in terms of the Optional criteria, it is not surprising that there 
is a great variety in what criteria are found in the district plans 
throughout the state. The four groupings chosen for analysis, 
however J attempt to cluster the district plans according to 1) 
whether schools compete with each other or must only meet some 
standard(s); 2) the degree of discretion held by an individual 
school to choose what Optional standards it must meet; and 3) the 
degree of difficulty a school has in meeting these standards. 

Grouping 1, labeled competitive, contains districts whose 
plans require competition among schools and a subsequent ranking 
according to the Optional Criteria standards chosen and met. It is 
the only one of the four groupings that reflects such competition, 
as the other three groupings require schools only to meet a 
predetermined standard. 

In the second grouping, labeled fixed by district, the 
Optional Criteria standards are determined at the district level, 
with no variation from school to school (within the elementary, 
middle or junior high school and high school classifications) . If 
a district's plan is placed in this category, all middle schools in 
that district, for example, must meet the proscribed standards. 

Groupings three and four actually represent different degrees 
of the same aspect of Optional Criteria standards, i.e., the degree 
to which a school is given discretion over which standards are to 



ERIC 



23 

be chosen. For grouping three, choice is minimal, while for 
grouping four a much greater degree of discretion is available. 

For all district plans, an initial determination of projected 
difficulty in meeting these Optional Criteria standards is made. 
The criteria used to determine degrees of difficulty are several. 
First, since there are 4 types of Optional Criteria standards 
identified in the legislation, and a potentially unlimited number 
of standards that could be chosen within each type, the total 
number of standards chosen must be considered. Although there may 
be exceptions, the fewer the standards that must be met, the easier 
it will be for schools to meet them. 

Second, the weight given to each standard varies from plan to 
plan. In some district plans, for example, each standard is 
weighed equally, as all standards mus-c be met and there is no point 
scheme. In other plans, there is a point rating scheme as, with a 
specified minimum number of points allocated to the meeting of each 
standard, and an overall minimum number of pclnts needed for merit 
school statu?;. 

A weighing scheme may not necessary help to decide if one set 
of Optional Criteria standards are easier than another unless the 
specific standards chosen are assessed according to how much effort 
beyond normal activities must be made by a school to meet them. It 
would seem likely that a standard th?.t read " a 1% increase in the 
number of students competing in the district science fair" would be 
much easier to achieve than, for example, a standard that wished to 
achieve "a 5% decrease in the school dropout rate". In the former 
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case, more teachers could simply make district science fair 
participation a course requirement. In the latter case, a 
specialized counseling program, among other activities, may have to 
be developed. 

In many cases, it seems as though districts have chosen 
Optional Criteria standards that only minimally meet the 
requirements of the legislation. In those districts, it seems 
likely that very few scliools who meet the statistically verifiable 
standardized test criteria are not given merit school status 
because they have failed or do not meet the Optional Criteria 
standards . 

An Initial Description of Optional Criteria Patterns, 1987-88 

Even though four categories were devised to describe the 
Optional Criteria, it is still very difficult to generalize across 
all district plans. It is additionally difficult to validly 
explain why such patterns exist without discussing the merit school 
program with district officials. Nevertheless, as indicated in 
Table 4, some initial description and analysis can be made. 

Only one district was found to have plans whose optional 
criteria clearly fell into category one (competitive) . In Dade, 
each school must create a school wide program based upon an 
identified theme. Each optional plan must contain elements such as 
a strategy for implementing the school wide plan and identification 
of how the results will be documented. 
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Over half (16 of 29) of the district plans identified optional 
criteria that were fixed at the district level without allowing for 
school discretion (category 2) • In many cases, the criteria seeaed 
easy to meet, requiring little additional effort by school teachers 
and administrators. Many of these plans identified only one to 
three criteria. One district, for example, requires only that 
school increase the number of students participating in the annual 
district art festival. Another district had only one requirement: 
increase student attendance. In a third district, schools must 1) 
increase the number of student entries in a school =inthology and 2) 
increase enrollment in advanced classes. In a Tourth district, 
three separate criteria are identified, one each for elementary 
schools, middle schools and high schools. 

Of the final two categories, which classify the degree of 
discretion given to individual schools, six districts received a 
rating of three (minimal discretion) , while four received a four 
rating (more than minimal discretion) . For both categories, 
schools were not competing with each other, but were attempting to 
achieve identified standards. Examples of district plans that are 
placed in these two categories include the following. One district 
directs schools to choose two standards from among a series of 
achievement and participation options offered. There are 11 
achievement standards, including "exceed by 2% the baseline 
percentage of students tested who perform above the 50th 
percentile". For participation standards, the school is directed to 
develop its own standards "with measurable criteria for 
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improvement" and an implementation plan. Suggested topics include: 
original writing from students, improving student attendance, 
increased parent participation, etc. Another district plan 
indicates that there must be at least a 1% gain in the number of 
students participating in approved competitive activities. There 
are over 200 activities listed from which a school can choose. 

In categoi-y 4, there is a greater increase in the school 
choice of optional criteria. For example, in one district's plan, 
each school has complete discretion over what will be chosen. 
Apparently the merit school committee in each school chooses a 
theme such as participation in a social science fair, a science 
fair, and a drop out prevention program* In this case, the choice 
of optimal criteria is decentralized to the school lev^l^ 

VThether or not a district's plan contains optional criteria 
that are judged to be easy or not, the plan can not identify the 
extent to which those criteria were met* Furtnermore, there can be 
no analysis of the effects of attempting to meet these standards on 
the overall goals of the merit school program. It may be, for 
example, that a school must only increase participation in the 
school writing anthology by one percent to meet the plan's 
requirements, but finds that it achieves a 20 or 30 percent 
increase. In doing so, it may also create a new sense of scnool 
pride and spirit that would not have otherwise existed- Additional 
school and district data must be examined to more fully ascertain 
the effect of this aspect of the merit schools program. 
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PATTERNS OF PARTICIPATION AND PERCENTAGES OF SCHOOLS IDENTIFIED AS 
MERIT SCHOOLS, 1984-1988 

As indicated in Table 5, several patterns of participation and 
designation of merit school status are observed when examining 
school districts data. Some districts designate a consistently low 
percentage of merit schools, while the percentage for other 
districts is consistently high. The percentage of merit schools 
has decreased for some districts, whi«3 it har increased for 
others. Possible res^sons for these patterns are discussed. 

Dade, Brevard and Putnam are three districts that have 
participated in all four years of the program, and consistently 
designate a 25-35% of scnools as merit schools. These results are 
likely to indicate a program established to discriminate among 
different levels of achievement. The program is designed to choose 
only the schools that score the highest on the designated criteria 
to be merit schools. After the first year of the program, in Dade 
county, for example, about two-thirds of all schools had 
participated in the program, with 25-35% of these achieving a merit 
school status- Dade created a two tiered system, with schools 
designated merit schools if they ranked in the top 25-35% on the 
standardized test criteria. To achieve a school of excellence 
designation, a school must create and implement a school wide 
program that is judged in competition with other schools . In 
Brevard, about 100% of schools have participated in the program 
each year, with 28-34% of the schools given a merit school 
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designation. 

Part of the explanation must lie in the fact that all three 
districts provide salary supplements (Dade and Brevard) and fund 
teacher awards (Putnam) (see table 6) as the means by which merit 
school funds are distributed. If the contract between teachers 
and the district designates a minimum amount for a salary 
supplement, e.g. $500 for a full time teacher, then this minimum 
may be the determining factor leading to a low percentage of merit 
schools in the district. 

Other districts have designated a much higher consistent 
percentage of schools as merit schools. Sarasota (4 years in the 
program, 51-60%), Volusia (2 years, approx. 54%), Pinellas (4 
years, 60-91%) and Orange (2 years, 81-94%) are examples of 
districts that fall into this category. In all four districts, 
almost 100% of schools participate in the program. There is no 
suggestion here that schools in these districts are somehow more 
meritorious than most of those in the three previously mentioned 
districts. It is much more likely that the standards chosen by 
these districts deliberately allow for a much higher percentage of 
merit schools. 

Patterns for other districts, though, are not consistent over 
time. In th<i first two years of the program, St. Lucie schools 
participated only at a rate of 35-37%, with 100% {n=7) winning 
merit school status in the first year, and 58% (7 of 12) designated 
merit schools in the second year. In the third year, no school 
participated in the program. In 198 7-88, 22 of 23 schools 
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participated, but only 6 (27.2%) became merit schools. A similar 
pattern has been evident in Lake, which did not participate in the 
third year as well. Even though participation of schools in the 
program has remained high (72-88%) , the percentage of schools 
designated as meritorious dropped from a high of 31% in 1985-86 to 
a low of 15% in 1987-88. 

Three districts have raised the percentage of merit schools 
quite drastically. Bay, after not participating in the program in 
the first year, reported that 90% of schools who entered the 
program received merit school status, a substantial increase over 
the 25% from the previous two years. All 15 of Martin's schools 
were merit schools in 1987-88, a percentage up from the 41-50% of 
the previous two years. The percentage of merit schools in Leon 
also increased from 77 in 1986-87 to 91 in 1987-88. 

A complete explanation for the inconsistent patterns can not 
be obtained from only examining plans and data available on file at 
the Department of Education. In theory, at least, some possible 
explanations can be proposed. These range from positive to 
cynical. 

If one purpose of the merit schools program is to raise the 
quality of instruction and increase learning in Florida's school 
districts, then is should not be surprising that percentages of 
merit schools would increase over time* Assuming that there are no 
logistical constraints that would prevent 100% of the schools from 
becoming merit schools in a given year, it would seem likely that 
within a very short time, teachers and administrators in a given 
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school would desire to achieve merit school status. This may be 
even more true if, for example, a rival school was designated 
aeritorious. Desiring the improved reputation that accompanies the 
merit school designation, all would work more diligently to 
increase student scores on standardized tests and the other 
Optional Criteria standards. 

A very sudden increase in the percentage of schools achieving 
merit school status, though, may mean district administrators have 
altered the choice of merit school standards to allow for a higher 
percentage of schools to win the merit school designation. It may 
be that both union officials and administrators have received many 
complaints from teachers who are in schools that have not been 
awarded merit school status 

From the analysis of the percentage of merit school winners 
over time, in most districts the criteria for determining merit 
school status have not changed. After the first year of the 
program, the change in the law allowing more than 25% of schools in 
a given district to become merit schools had some impact on the 
acceptance of the program by some districts. Those districts that 
have entered the program more recently have tended to choose 
criteria that lead to a high percentage of merit schools. As cur 
content analysis of district plans indicated, in many cases 
districts used the same plan year after year, changing only the 
dates. 

DISTRIBUTION OF FUNDS RECEIVED BY SCHOOL DISTRICTS 
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Each district may choose to spend merit school funds in 
several different ways. There are three general categories: 1) 
salary requirements; 2) materials; and 3) individual awards. 
Within these three categories/ variation can exist. Salary 
supplements may be a variety of amounts, and can be allotted to 
administrators, teachers / and other school personnel at the 
district's discretion. Materials can be purchased by each school's 
QUIIP committee, or an individual teacher may be allowed to decide 
what materials should be purchased to be used only for his/her 
classes. The legislation does identify individual award 
categories, including 1) outstanding (teacher) attendance; 2) 
employment in a critical teacher shortage subject area; 3) 
employment in a critical teacher shortage school site; 4) 
reimbursement for completion of in field credit; and 5) superior 
performance evaluation. Districts which choose the individual 
award category, however, are not limited to these legislative 
suggestions and may create their own programs. A district may 
choose one or more of these categories, with a variety of fund 
dollars allocated to one or m.ore category. The only legislative 
stipulation is that at least 50% of the funds be allocated to the 
schools (or teachers and other personnel within the schools) that 
are designated merit schools. 

Table 6 indicates the percentages of fund distribution for all 
participating districts during the first four years of the program. 
Very few districts that have participated throughout the program 
have changed the categories in which they allocate the funds. Some 



TABLE 6: Distriixitlon of Herit Funds 



(10/30/89) 





1984 • 


1985 


1985 • 


t986 


1986 • 


1987 


1987 • 


1988 


County 


Area 


Amount 


Area 


Arnount 


Area 


Amount 


Arta 


Amount 




8SS8S83SS8SSasSSS8S8SS883 


8S8^SaSittettS88&8CCe8S88SS3 


S8SS88SSCSS888a88SC8SSaS8 


8888888S88888888S888SS: 




6 ♦ H 




i 6 ♦ H 


S0X*50X 


6 ♦ H 


50X450X 


6 4 H 


50X*50X 


Say 


... 


... 


1 6 


96X 


6 4 7 


76X09X 


6^7 


73X*19X 


•rtdford 


... 


... 


1 6 ♦ H 


52X^48X 


6 4 H 


UX4 50X 


H 


100X 


Irevard 


6 ^ H 


89x*in 


6 ♦ H 


69X430% 


6 « H 


61X438X 


6 4 K 


55X445X 


Iroward 


6 


99X 


6 


98X 


... 


... 


• •. 




Calhoun 


1 




i 6 


100X 




100X 


1 6 


100X 


Cotunbia 


6 ♦ H 


50X^505C 


1 6 ♦ H 


50X450X 


... 


... 


1 ... 


... 


Dade 


1 6 ♦ H 


66X*32X 


1 6 


97X 


1 6 


96X 


6 


99X 


OeSoto 


1 6 


99X 


1 


... 


1 


... 


1 ' ** 


... 


Dixie 


6 


99X 


1 


... 


... 


... 


1 


... 


Culf 


7 


99X 


1 7 


99X 


1 


... 


7 


98% 


riagter 


1 


... 


1 


... 


7 


10CX 


1 ^ 


100X 


Hami Iton 


1 6 ♦ H 


84X*15X 


1 ^ 


10CX 


1 


... 


1 *** 


... 


Hardee 


1 6 


99X 


1 6 


98X 


1 


... 


1 6 4 H 


52X4A5X 


Highlands 


1 6 


9ex 


1 


... 


1 


... 


1 **' 


... 


Indian Rwer 


1 6 ♦ H 


50X»i9X 


1 6 ♦ H 


50X*5CX 


1 6 ♦ H 


50X*50X 


1 6 4 H 


57X*A3X 


Jackson 




24*2«.*A7X 


1 


... 


1 


... 


1 *'* 


... 


Lafayette 


1 6 ♦ M 


50X*5CX 


1 


... 


1 


... 


1 ... 


... 


Lake 


1 6 ^ K 


49X*50X 


j 


37*12*5CX 


1 ... 


... 


1 6*7*H 


3802*5C 


Leon 


1 


... 


1 


... 


1 7 


99X 


1 7 


100X 


Levy 


1 6 ♦ H 


64X*35X 


1 


... 


! 


... 


1 *" 


... 




1 6 « H 


r9X*2CX 


1 6 ♦ K 


65X*35% 


1 6 * H 


62X*3n 


1 6 4 H 


b9X*;0X 


Manatee 


1 6 ♦ H 


50X*5DX 


1 


... 


1 


... 


1 — 


... 


Marion 


1 


... 


1 


... 


1 


... 


1 7 


99X 


Martin 


1 6 ♦ H 


8ax*i2x 


1 6 ♦ h 


78X*22X 


1 6 4 H 


49X*A9X 


1 6 4 44 


5CX450X 


Monroe 


1 6 


93X 




91X 


1 6 


92X 


1 6 


90X 


Na&sa. 


1 6*7 


74X*2CX 


! 6*7 


75X*20X 


! 6 


95X 


1 6 


9AX 


Orange 


1 




1 


... 


1 7 


100X 


1 7 ♦ H 


69X*26X 


Pascc 


1 6 « H 




1 6 ♦ H 


51X^49X 


1 6 * h 


50X*50X 


1 6 4 H 


A9X*4,8X 


Pinel las 


1 7 


10CX 


1 7 


100X 


1 7 ♦ H 


95X*5X 


1 6 4 H 


95X*5X 


Putnam 


i 6 


99X 


1 6 


99X 


1 6 


100X 


1 6 


99X 


St. Lucie 


1 6 


100X 


1 6 


lOOX 






1 6 


100X 


Sarasota 


1 6 ♦ H 


70X>23X 


1 6 « H 


65X*22X 


1 6 4 M 


73X03X 


1 6 4 H 


7i;X*7X 


Seminole 














1 7 4 H 


71X>20X 


Sijnter 










1 6 


100X 


i 6 


100X 


Suwannee 


1 6 ♦ H 


49X-^48X 


1 6 ♦ H 


50X450X 


1 6 4 H 


57X440X 


1 6 4 H 


5AX4A6X 


Union 


1 6 


100X 


1 6 


100X 


i 7 4 H 


66X433X 


1 6 


100X 


Volusia 














1 7 ♦ H 


71X428X 


Ufikutla 


1 6 


10CX 


1 7 


100X 




. • . 






Washington 


1 7 


99X 


1 7 


100X 






1 ^ 


100X 



COOES 



6 * salary supplements 7 * naterials H * Individual awards 



^8 



TABLE 7 
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exceptions are Bradford, which shifted frora a combination of salary 
supplements and individual awards to 100% individual awards; and 
Hardee, which shifted from nearly ell salary supplements to a 
combination of salary supplements and individual awards. 

When the totals for each categoi-y ar3 compared, however, one 
interesting pattern does appear. As shown in Table 7, in 1984-85, 
25 or 30 (83%) of districts chose to distribute the funds directly 
to individual personnel (6 and 6+H) , with only 17% of the dirtricts 
providing any money for materials. By 1987-88, the former category 
had fallen to 62%, with an increasing number of districts (34%) 
providing some funds for materials. Those districts which have 
entered the merit schools program in the last few years have thus 
tended to emphasize the entire merit school receiving funds for 
materials rather than individuals receiving funds. 

Comparison of Fund Distribution by Percentage of Merit Schools 
Within the District 

For 1987-88, Table 8 provides additional descriptive analysis. 
When categories of fund distribution are also categorized by 
percentage of merit schools within a district, some additional 
patterns are evident. First, no district that provides only salary 
supplements (category 6) has designated more than 57% of its 
schools as merit schools. Although further examination is 
necessary to fully explain this result, it seems likely that a 
minimum dollar amount per K.erit school employee has been 
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established • Depending upon the total funds available to a 
district, this laininium amount would also limit the number of 
schools designated as meritorious. 

Given the distribution process at the district level for funds 
to be spent on materials (category 7) , it is not surprising that 
the percentage of merit schools for most of these districts is 
high* In these latter districts, a school based merit school 
committee is likely to decide what materials will be purchased with 
these funds. There is no concern that some minimum amount per 
teacher needs to be awarded, as there may be in those districts 
that distribute the funds through salary supplements. 

THE MEASUREMENT AND DOCUMENTATION OF MERIT SCHOOL GOAL 

ACHIEVEMENT 

In most districts, school buildings are required to keep on 
file documentation supporting the achievement of merit school 
goals. For the most part, this documentation refers to the 
attainment of the optional criteria, as standardized achievement 
test scores are collected at the district level. In many cases, 
buildings report to the district the relevant percentages and 
supply a narrative description of the activities that lead to the 
achievement of the optional criteria. 

Interview data indicated, however, that little if any 
accountability is required. No one from the district level in many 
districts ever requests buildings to provide the supporting 
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documentation for the level of achievement claimed. In essence, 
the word of the principal or school administrator is accepted as 
the truth. 

There was no indication ^ however, that schools cheated in 
their attempt to score well on the optional criteria. This 
behavior would not be occurring in those districts in which a high 
percentage of schools were awarded merit school status. In other 
words, cheating would not be necessary. In a hiqh percentage of 
cases, schools did not achieve merit school status bec-^use their 
standardized achievement test scores were not high enough. For 
these schools, supporting documentation for the optional criteria 
would not be needed as well. 
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THE MERIT SCHOOLS PROGRAM: IMPACT ANALYSIS 

This section of the finsl report deals with general or 
overall programmatic impacts and more specific impacts of the 
merit schools program, including : 1) oocioeconoiaic level and 
minority patterns among students; 2) student learning; 3) 
instructional practices; 4) school based management; 5) economic 
incentives for teachers; 6) a sense of competition; and 7) 
personnel practices. There are two data sources. A survey 
instrument mailed to a sas^ple of teachers, principals and school 
administrators, and district administrators comprises the primary 
data source. Second, to analyze the impact of the program in 
terms of the socio-economic composition of the students, free and 
reduced lunch data and ethnic group membership from 1936-89 was 
obtained. 

Results of the survey contribute to the impact analysis in 
several respects. First, agreement with specific sets of 
statements were combined to form three indices. Using analysis 
of variance, results indicate that there were no significant 
differences among the responses from elementary schools, 
middle/ junior high schools, and high schools. When districts are 
divided into categories according to how merit dollars are 
awarded, significant differences do occur. In other words, 
responses from districts which award merit school funds primarily 
in the form of instructional materials and equipment differ 
significantly from those districts which provide primarily salary 



36 

supplements. For reasons that will be discussed, responses from 
Dade County constituted a third category. Responses from 
teachers, principals and other school administrators are analyzed 
according to the three categories of district? » In this 
analysis, responses to specific statements are examined. 

Using the above mentioned data, we first assess the overall 
impact of the merit schools program. After reviewing the impact 
of the program in terms of student socioeconomic Ipvel and 
minority patterns, we then discuss the more specific impacts on 
student learning and instructional practices. Within the 
discussion of student learning, a comment on the use of 
standardized tests and the statistical model requirement of the 
program is included. Another positive impact of the program, the 
encouragement of school based management practices, is then 
examined* Next, two more minor impacts, those dealing with a 
sense of competition and with teacher transfers, are discussed. 
We then analyze the program in terms of whether it provides 
adequate economic incentives for teachers before providing our 
concluding comments which include recommendations for future 
changes . 

SAMPLING PLAN FOR 
SCHOOL ADMINISTRATOR AND TEACHER QUESTIONNAIRES 

A two stage sampling strategy for school administrators and 
a three stage sampling strategy for teachers was used. The 
stages were: 



37 

!• Stratified random sample of participating 
districts (with two modifications) 
2. Stratified random scunple of schools within 
selected districts 

3. Systematic random sample of teachers within 
selected schools 

Stage 1: Stratified random sample of participating districts 

To ensure representativeness, a decision was made 
to survey one third of the participating districts. 
Districts which participated in 1987-8S were classified 
into a 5 X 3 table- The five levels of the first 
variable were types of awards; salary supplements only, 
salary supplements and individual awards, materials 
only, materials and individual awards, other plans. The 
three levels of the second variable were percent ranges 
of participating schools classified as merit schools; 0% 
to 35%, 36% to 70%, 71% to 100%. 

Modification : Because Dade county received approximately 
one third of the entire lerit Schools budget and included 
approximately one third of the participating schools 
within the state, Dade County was selected tc 
participate and was removed from the matrix of remaining 
districts. 

The three cells associated with "other plans" were 
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combined. There were only two districts, one in each of 
two of the three cells. One of the two was selected by 
use of a random number table. 

Nine additional districts were selected in the following 
manner. Nine of the remaining twelve cells contained school 
districts, 2 cells with four districts, 4 cells with 
three districts, 2 cells with 2 districts, 1 cell with 
one district. With the exception of the 1 district cell, 
districts were randomly assigned a number from one to 25 ♦ 
A series of random numbers was generated and used to 
select one school district from each of the eight cells 
containing two or more districts. The district in the 
one district cell was also selected. 

Modification : because the Merit Schools program is a 
state-wide program and districts from every region of the 
state participate, the selected sample of districts was 
checked for regional representation. Two regions were 
under represented, the north west panhandle and the west 
central regions- One additional district was randomly 
selected from each of those regions. A total of thirteen 
districts were included in the sample, 

Staje 2: Stratified random sample of schools within selected 
districts 
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The schools within each district which participated in the 
merit schools program for one or more year were stratified by 
level: elementary school, middle or junior high school, high 
school. Special schools and alternative schools were excluded. 
The Florida Education Directory, 1988-89, was used as the 
reference. For each selected school district, the schools within 
each stratum were numbered alphabetically. An independent set of 
random numbers was generated for each set of schools. A twenty 
five percent random sample was selected from each set. At least 
one school was selected from each stratum for each school 
district. 

The resulting sample consisted of 180 schools; 118 
elementary schools, 37 middle or junior high schools, and 25 high 
schools. Each school with less than 75 teachers was sent two 
school administrator questionnaires; ont* for the principal and 
one for any other administrator in the school. Each school with 
7 5 or more teachers was sent three school administrator 
questionnaires, one for the principal and two to be distributed 
to any other appropriate administrators. 

Stage 3: 3/stematic random sample of teachers within selected 
schools 

The desired number of teacher returns was set at 1,000. 
Using an expected return rate of apprc.imately one third, 3,000 
teacher questionnaires would need to be distributed. 
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Approximately 9,000 teachers were employed in the 180 selected 
schools. A distribution of 3,000 questionnaires among 9,000 
teachers, required that the one third of the teachers be given 
questionnaires . 

The principal in each selected school was sent a packet of 
containing questionnaires for one third of the teachers assigned 
to the school. The principal was asked to consider the list of 
teachers to be numbered and to distribute the questionnaires to 
every third teacher beginning with teacher number three. Two 
forms of the teacher questionnaire were created. Copies of form 
A and form B were alternated. Every sixth teacher received form 
A and evary sixth teacher received form B. 

INDEX CREATION 

To help assess the impact of the program, three indices were 
created. Each represents a general measure of the three major 
impacts of the program. ATTITUDE reflects the overall general 
feeling about the merit schools program. INSTRUCT measures the 
extent to which a major goal of the program has been achieved, 
the improvement of instructional practices. TEAMWRK represents 
the degree to which school based management activities and 
practices were increased because of the program. 

Each index is an additive scale formed from responses to 
various statements. ATTITUDE reflects responses to ten 
statements, five of which are phrased in a format favorable to 
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TABLE 9 



QUESTIONNAIRES MAILED 



DISTRICTS AND NUMBER OF SCHOOL 
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TABLE 10 



Materials and Equipinent Districts; 
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Middle/ Jr. High 




5 


of 


21 


Elementary 


Washington 


1 


Of 


2 


High Schools 




1 


Of 


2 


Middle/ Jr. High 




1 


Of 


2 


Elementary 


Volusia 


1 


Of 


7 


High Schools 




2 


of 


8 


Middle/ Jr. High 




9 


of 


35 


Elementary 


Orange 


2 


of 


10 


High Schools 




4 


of 


19 


Middle/ Jr. High 




18 


of 


73 


Elementary 


Leon 


2 


of 


5 


High Schools 




2 


of 


6 


Middle/ Jr. High 




4 


of 


20 


Elementary 


Pinellas 


3 


of 


15 


High Schools 




4 


of 


21 


Middle/ Jr. High 




18 


of 


74 


Elementary 



o 
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TABLE 11 



QUESTIONNAIRE RETURN RATE 

1. District Coordinators: 26 of 29 

90 % 

2. School Administrators: 

High School Middle/ Jr. Elementary 

Dade 10 of 18 16 of 27 46 of 89 

56% 59% 52% 



Salary Supplement 14 of 21 15 of 20 21 of 38 

67% 75% 55% 



Materials and Equp 20 of 31 13 of 32 55 of 111 

65% 41% 50% 



3 . Teachers ; 



High School Middle/ Jr. High Elementary 



Dade 76 of 275 76 of 270 249 of 667 

28% 28% 37% 



Salary Supplement 62 of 209 41 of 129 103 of 206 

30% 32% 50% 



Materials and Equp 126 of 387 

33% 



91 of 269 
34% 



287 of 655 
44% 
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the merit schools program, and five of which are negative. 
Scaling of the negative statements was adjusted prior to 
addition. In the former group are statements suggesting that the 
program is fair to all schools, has a positive impact, rewards 
good work, results in a wise use of state funds, and helps school 
improvement. In the more negative group are found statements 
such as the merit schools program has pushed schools towards 
trivial learning, promotes practices that diminish learning, uses 
test scores that give false information, is influenced by 
politics, and results in a competition that is corrupting. 

INSTRUCT is the compilation of reactions to four statements 
concerning instructional practices. These are that the program 
requires teachers to contribute more hours to school, results in 
more time spent on student learning, encourages teachers to use 
more effective instructional methods, and produces improved 
equipment and materials. 

TEAMWRK measures the degree to which the program has 
improved practices that lead to a greater acceptance of school 
based management practices. Responses from seven statements are 
compiled, reflecting the degree to which Lhe program has resulted 
in better teamwork among teachers, among teachers and 
adminiiitrators, and five statements concerning increased teacher 
input into decisions about the choice ol instructional methods, 
the purchase of equipment and materials, school policy and 
procedures, school goals, and activities found in other grades. 
In terms of the district categories, survey responses were 
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separated into three categories, based upon the method of 
awarding merit school funds. These categories are: a) respunses 
from those districts that award salary supplements; b) from those 
districts that award dollars to be spent primarily on 
instructional materials; and c) from Dade County. Dade County 
was separated because: 1) it has the largest merit school 
program; and 2) although it distributes its award dollars in the 
form of salary supplements, responses from Dade educators 
indicate a much more positive view of the merit school program 
than is found in other districts that award salary supplements- 

When examining the means for each district category by 
school level for each index, a clear pattern emerges (see tables 
in Appendix H) . When means are reviewed and a two way analysis 
of variance performed, there are significant dif f erenc^.s among 
district categories. In every case but two, however, there are 
no significant differences among school levels. In no case is 
there a significant interaction effect. 

When examining the ATTITUDE index for the first teacher 
survey, by examining the means for districts, and comparing the 
difference among the means as a fraction of the pooled standard 
deviation, a practical difference in significance appears for 
district categories. The difference between the means for salary 
supplement districts and materials districts is 4.5, an amount 
almost equivalent to one half the pooled standard deviation. The 
summary means for school levels, though, are much closer, with 
the difference between the lowest and the highest only 1.7. As a 
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fraction of the pooled standard deviation, this figure is not 
high enough to be considered significant. For the second 
teachers survey, the results for the ATTITUDE index are very 
similar, with means for districts ranging from 30.6 to 34.0 ( 
difference of 3.4 — more than one third the pooled standard 
deviation of 9.08), and means for school level ranging from 32.7 
to 33.2. For school administrators, the school ],evel means 
difference of 3,1, and the district mean difference of 5.2 both 
indicate a practical significant difference given the pooled 
standard deviation of 9.26, 

The INSTRUCT index reflects a similar pattern, with salary 
supplement district teachers providing a smaller overall mean of 
12,3, much less than the high of 14.3 for materials only 
districts. The difference in means is 2,0, more than one half 
the pooled standard deviation of 3.6* As school level mean 
differences are only .9, one quarter of the pooled standard 
deviation, there is no practical significant difference. For 
school administrators, school level means are almost exactly the 
same (12.7), while the same pattern holds true for districts as 
did for teachers, with a mean difference of 2.8 compared to the 
pooled standard deviation value of 3.8-*. 

Results for the TEAMWRK index reflect the same pattern. 
Difference in district means for teachers is 2.9, almost one half 
the pooled standard deviation value of 6.54. The school level 
difference of 1.7 is less than one third. For school 
administrators, the practical significant difference for 
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districts is even greater, with a mean difference of 4.9 compared 
to a pooled standard deviation value of 6.61. The school level 
mean difference of 1.2 reflects no practical significant 
difference. 

With the indices providing guidance concerning how to 
present our analysis, the next section provides a more specific 
examination of the responses to various statements for each major 
impact area. Our analysis will refer to the responses found in 
Appendices X & Y, with page and statement noted. Responses 
indicating "strongly agree" and "agree" have been collapsed into 
an "agree" category; similarly, "strongly disagree" and 
"disagree" responses have become "disagree". Throughout, 
responses from teachers and administrators will be compared, as 
well as responses from the three district categories* 

OVERALL IMPACT OF THE MERIT SCHOOLS PROGRAM 

In general, the impact of the merit schools program is 
viewed as positive, with principals and school administrators 
having a more positive view than teachers. Responses from the 
materials only districts are much more positive than those from 
the salary supplement districts, with responses from Dade 
positive but less so than the materials only group. In fact, 
responses to the statements that comprise ATTITUDE fall into 
three groups: 1) a majority of all three districts agree; 2) a 
majority of the inst/materials and Dade districts agree, while a 
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minority of salary supplement districts agree; and 3) mixed 
results, with a significant percentage from all three district 
categories choosing undecided. 

Asked to agree or disagree with the statement, "the merit 
schools program has a positive impact on my school", 80% of 
principals and 72% of teachers from materials only schools 
agreed; in Dade county, from participating schools the 
percentages were 71 and 61 in agreement, respectively, while the 
remaining salary supplement schools indicated a 56% and 54% 
agreement (A-4-b; B-4-b) . Reward for good work is occurring, 
although Dade County responses are more positive (70% teachers 
and 7 3% principals in agreement) than other salary supplement 
districts (57% and 50% in agreement) and the materials districts 
(44% and 67% in agreement) (A-4-c; B-4-c) . School administrators 
overwhelmingly disagree that the program has had a corrupting 
influence, as percentages range from 67-77 {B-5-n) . Also, a 
majority of administrators feel the program does not push schools 
towards trivial learning (58-71%) (B-4-d) • 

Responses to other statements, though, clearly reflect the 
more negative attitude held by teachers and principals in the 
salary supplement districts than by their counterparts in the 
other two district categories. A majority of teachers and 
administrators in Dade feel that the program is " a credible one 
for school improvement" (57% and 62%), as they do in the 
materials only districts (55% for both). In the salary supplement 
district's, however, the agreement is much lower (40% and 38%) (A- 
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5-1; B-5-1) . Similarly, a majority of administrators in Dade and 
materials only districts feel the program reflects a "wise use of 
state funds" (52% and 55%), while their counterparts in s«xiary 
supplement districts do not agree, with only 26% in agreement (A- 
5-k; B-5-k) . 

A third category of responses suggests a mixed, undecided or 
negative view. A majority of administrators in materials only 
schools agree that the program is fair to all schools, while a 
majority in the salary supplement schools disagree (52% in both 
cases) For Dade, the results are mixed, with 42% in agreement and 
38% in disagreement (B-4-1) . Among teachers, the same stateuent 
also produces mixed results, as agreement percentages range from 
25-41 and disagreement percentages range from 25-49 (A-4-a) . For 
both teachers and administrators, salary supplement districts 
were more negative, with materials only districts the most 
positive. 

Overall, a high percentage disagree that the program should 
be dropped, with percentages ranging from 43-72% (A-5-o;B-6-o) , 
but teachers, more so than administrators, feel that some 
modification is needed (teachers: 47-62% in agreement; 
principals: 27-33% in agreement) (A-5-p;B-6-p) . 

District Administrators support the general positive view of 
the program, with 46% agreeing that teachers feel positively 
towards the program (D-1-4) • Similarly/ 62% agree that their 
district's program is working well (D-1-6) . In response to the 
statement that the program "has ^aded little to the activities 
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and behaviors that teachers and principals typically perform in 
our district", 54% disagreed (D-1-2) . 



IMPACT OF SOCIOECONOMIC LEVEL AND MINORITY MEMBERSHIP PATTERNS CF 
MERIT SCHOOLS 

The state legislature stipulated that the opportunity to 
earn merit school status should not be related to ethnic make-up 
or socioeconomic status of the students attending schools. To 
answer the question of whether the patterns of merit school 
winners in any way is reflective of the socioeconomic status or 
percent of minority students in a school, we examined the 
reported data of individual school winners provided by each 
district and the DOE annual report of racial/ethnic category and 
economic status by school within district. Following the 
procedure used by DOE, socioeconomic status (SES) was determined 
by the percentage of enrolled students that are eligible for free 
or reduced lunv^n. 

The percentages of students classified as aimority and the 
percentage of student eligible for free and reduced lunch vary 
greatly from district to district. In addition, the percentages 
of students that are eligible for free and reduced lunch varies 
by school level. The percentages are always higher in elementary 
schools and lowest in high schools. For these reasons, the data 
had to be reviewed independently for each aistrict and for each 
school level within a district. 
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Because of the different nuiaber of schools in districts of 
different sizes, the patterns were reviewed differently for very 
small and small school districts, moderately small districts, and 
large and moderate sized districts (see tables in Appendix I) . 
The method of review also varied by level of school because of 
changes in the number of schools* 

In several cases, schools were ranked vithin level of school 
and district, SES rank was based on the percentage of students 
qualifying for free and reduced lunch with the lowest- percentage 
given the rank of 1 and the highest percentage given the greater 
number; for example, 6th of 6 schools. Rank in percentage 
minority was based on the percent of other than white (non- 
hispanic) as reported by DOE. The school with the lowest 
percentage of minorities was given the rank of 1. The school 
with the highest percent of minorities was given the larger 
number; for example, 15th of 15 schools. 

In the very small and small school districts, winning merit 
school status seems to be shared equally among the schools at 
each level. The exception would be Putnam counry for which the 
pattern reflects no consistent bias. 

In the moderately small county schcs^i districts, the 
patt-erns show very little systematic bias. In Leon and Marion 
counties, however, the elementary schools with higher percentages 
of students that qualify for free and reduced lunch and those 
with higher percentages of .ninority students tend to win merit 
scbool status slightly less often. There may be some other 
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explanation for these weak relationships. 

We obtained data for some of the larger districts. Seminole 
and Orange counties award high percentages of schools every year 
and there were no consistent patterns that suggest bias. We did 
not have a list of winning schools provided by Pinellas county, 
but they awarded most all of their schools every year and thus 
there WdS little opportunity for bias. There was no evidence of 
bias in the data from Pasco county. The data from Brevard and 
Volusia counties indicate that the distribution of awards to 
schools in those counties were slightly in favor of the schools 
with higher SES and lower percentages of minority students. But 
in both cases, the relationships are very weak overall, although 
they are more evident at one level of school in each county. 
Again, there may be a number of other reasons for the observed 
results • 

For the most part, the evidence suggests that there is a 
weak relationship between SES status and percent minority and the 
likelihooa of winning merit school status. The pattern is not 
consistent from school district to school district. In several 
districts there is a tendency for schools with low^r SES (higher 
rank numbers) and larger percentages of minority students (higher 
rank numbers) to win merit school awards less often, while in 
others the pattern is reversed and in others there seems to be no 
pattern at all. 

As can be expected, in those districts that award a very 
high percentage of schools as merit school every year, there is 
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no relationship. The tendency for a relationship between merit 
school winners and SES or percentage minority has a greater 
potential for being exhibited in districts that award smaller 
percentages of schools each year. For the most part, it is in 
those districts that the weak relationships can be observed. 

INCREASED STUDENT LEARNING 

Our analysis indicates that the program is perceived as 
having a positive impact on student performance, although the 
extent to which this occurs varies from school to school and from 
district to district. Many educators believe that the activities 
required by the ir'^rit schools program, including those leading to 
increased standardized test scores, would be occurring in the 
absence of the program. Yet other educators believe that the 
program sensitizes teachers and administrators toward 
accomplishing those activities more fully than if the program did 
not exist. The fact that there is a visible reward for 
increasing test scores and/or increasing the percentage of 
students that pass a physical fitness test, for example, does 
make a difference in the amount of time and attention given to 
these activities. Furthermore, that the merit schools program 
requires that teachers choose these activities through a 
participatory process further ensures more successful 
accomplishment. 
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Standardized Achievement Test Scores 

The current requirement that meritorious schools must 
demonstrate student achievement above an expected level on a 
standardized achievement test has not been successful. The 
requirement is based on the false assumption that student 
performance or academic achievement and standardized test scores 
are synonymous. Student academic ability and standardized test 
scores are often very different constructs. 

It is easy to increase test scores without improving student 
learning. Once educators learn that their work is to be judged 
almost entirely by scores on such tests, they devise methods that 
result in higher test scores. Sometimes the test score rather 
than student learning becomes the major goal. The distinction 
between instruction on test taking skills and teaching the test 
may b'^come blurred. Test companies add to the problem by 
convincing schools that the items on a norm-referenced 
standardized test can serve a criterion-referenced function. 
Classroom lists of items missed by students this year indicate to 
teachers which items to focus on for the following year. It 
becomes a wajor problem when copies of the tests are available to 
anyone who makes an effort to obtain a copy. 

It i s now well understood that high stakes pressure to 
increase scores on standardized tests results in invalid scores. 
The test items change only once in every five to ten years. Over 
time, the test items become a major part of the formal and 
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informal curriculum. The norm based scores become meaningless* 
To the extent that a given district equates performance 
with test scores, and there is a high degree of competition to 
become .a merit school within that district, a situation of "high 
stakes testing pressure" may result. This is one reason the 
responses to the statement "student learning has improved" elicit 
a much lower agreement from educators from the other salary 
supplement schools than from educators from the materials only 
schools. In the former, only 37% of teachers agree, and only 32% 
of principals agree (A-l-a;B-l-a) . For the latter group, 50% of 
teachers and 54% of principals agree. Additional insight is 
provided by the similar pattern of responses found to two other 
statements. A minority of teachers and principals from the other 
salary supplement schools agree that "more time is used for 
learning activities" (40% and 25%, respectively) (A-l-c;B-l-c) , 
while the percentages of agreement are much higher in materials 
only schools (51% and 41%). In contrast, more teachers and 
principals from the salary supplement schools agree that "more 
time is used to teach test taking skills" (69% and 65%) than in 
the materials only schools (55% and 44%) (A-2-j ;B-2-j) . These 
results seem to indicate that more pressure on the standardized 
tests is closely related to less student learning. 

Even though Dade County distributes merit school awards in 
the form of salary supplements, the responses from teachers and 
administrators are much l ^re positive than their counterparts in 
other salary supplement schools. In fact, 52% of teachers and 

Er|c 74 



53 

53% of principals from participating schools agree that student 
learning has improved. In part the explanation for this result 
may be that the Dade County program is unique in that it requires 
each participating school to develop a year long school 
improvement plan. In doing so, it broadens the definition of 
student learning to include these criteria. Student learning is 
improved when, for example, the entire school works on a year 
long world geography project. 

In contrast, in many of the other salary supplement 
schools, these other criteria are not emphasized nor given much 
importance in terms of how they increase student learning. 
Increasing the student participation in a science fair, for 
example, may be accomplished by simply assigning the project as a 
course requirement for a larger number of students than the 
previous year. 

The Dade County program, though, does have unique problems. 
While much of the positive benefit is nrobably related to the 
year long school improvement projects, merit schools are selected 
only on the basis of increases in standardized test scores. 
Teachers and principals are not aware of the procedure used to 
review test scores and receive no feedback about their relative 
standing on the test score improvement. One possible additional 
problem is reflected in the Dade administrator response to the 
statement, "teachers tend to transfer from other schools to 
schools that often earn merit school awards". The percentage of 
agreement from Dade was 24%, much higher than responses from the 
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Other categories of districts: 4%'-8% {B-2-o) . 

Another set of potential problems were illustrated by 
conunents from teachers in a small sample of schools we visited. 
In one of the schools, teachers believed that merit schools were 
selected on the basis of the presentation of their special 
program and did not know that test sores even counted. In 
another school, teachers worked very hare on teaching test taking 
skills and other activities related to improving test scores 
every year in order to ensure that test sores rose so the school 
would earn merit school status. The teachers in the latter 
school did not put a major effort into their year long program 
every year. They varied the magnitude of their efforts from year 
to year in order to avoid teacher burn out. They believed that 
the extra $500 above the first $500 merit school award was only 
worth the effort every few years. 

Because of the lessened emphasis on standardized test 
importance in the materials only schools, the definition of 
student learning is expanded. Even if the materials schools do 
not choose a school wide project, they are more likely to 
appreciate the benefits that additional materials and equipment 
(e.g., new computer hardware, science materials, library 
materials, etc) have for student learning. It is also more 
likely that teachers and principals in these schools will 
understand the connection between increasing the number of 
volunteer Amours, for example, and increased udent learning. 
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The Statistical Model Requirement 

In spite of many mathematical and educational problems 
associated with assessing school performance with standardized 
achievement scores, some districts are satisfied with the models 
they have developed to fulfill the current standardized test 
score requirement of the program. There are serious mathematical 
and educational problems with most all of the models that are in 
current use. For example in districts that compare student scores 
from year to year, there is seldom any consideration of schools 
that have for many years produced students who score above what 
can be expected and thus are meritoricus by definition. Such 
schools may not achieve merit school status because they are 
already acnievihsj the desired goal and have not demonstrated even 
greater performance improvement. Some districts recognize these 
problems but are willing to live with them in order to 
participate in the program.. Other districts have experienced 
serious problems in creating and implementing appropriate models. 

The fact that most every district has developed a unique 
model speaks to the lack of a clear acceptable strategy for 
establishing appropriate expected levels of performance on 
standardized achievement test. Unfortunately, the strategies 
used in some districts that reduce problems associated with 
ceiling effects and a constant need to produce scores even above 
indicators of excellence have not been shared among districts. 
Without sharing of experiences and strategies, grossly 
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inappropriate models have been adhered to because of problems 
related to new negations with bargaining units and approval by 
the DOE. 

IMPROVED INSTRUCTIONTO. PRACTICES 

Th*^ composition of the index INSTRUCT reflects four 
statements that measure, in different ways, the extent to which 
instructional practices are improved because of the merit schools 
program. These statements focus on the materials ixnd equipment 
used for instruction, the effectiveness: of the instructional 
practices used, whether the teachers devcte more time to school, 
and whether, as a result, more time is spent devoted to student 
learning. 

Of the four statements, there is only overwhelming agreement 
for improved equipment and materials, and this is only from the 
materials only districts (teachers: 72% agreement; 
administrators: 86% agreement). This result is clearly 
understandable, as in these districts there is a clear linkage 
between the program and the purchase of additional equipment and 
materials. Likewise, it is not surprising that responses from 
the other districts do not share this level of agreement, since 
merit school funds are awarded as salary supplements. Neither a 
majority of teachers nor administrators in either Dade or in the 
other salary supplement districts agree, with percentage 
agreement ranging from 32-44 (A'-2- ;B-2-h) . 
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To the extent that improved equipment and materials results 
m teachers using improved instructional practices, there is some 
indication that this is occurring in the materials only 
districts, as 61% of teachers and 57% of administrators agree 
more effective instruction is occurring. The responses from the 
other two district categories are not as clear cut. Although 45% 
of teachers from Dade agree, 35% of them are undecided. It is 
not clear in the minds of over one third of the Dade teachers 
whether or not the merit schools program results in the adoption 
of more effective instructional practices. Educators in the other 
salary su^^^lement districts are surprisingly split, as more 
administrators disagree than agree (41% to 37%) , while more 
teachers agree than disagree (47% to 26%) (A-l-d;B-l-d) . The 
responses from the District Administrators support this mixed 
view. Although 4 6% agree with the statement that the program "is 
achieving its primary goal of improving instructional practices 
in our district", a significant 31% disagree (D-1-3) . 

When asked to respond to the statement "more student time is 
used for learning activities", the patterns of responses are 
similar, with educators from the materials only districts more 
positive than negative (teachers: 51% agreement; administrators: 
41% agreement). Likewise, Dade educators show a positive response 
(teachers: 47% agreement; administrators: 49% agreement. Again, 
administrators in the other salary supplement districts are more 
negative than positive (46% disagree to 25% agree) , while 
teachers responded in the opposite fashion, with 40% agreement 



ERLC 



79 



58 

and 30% disagreement (A-i-c;B-l-c) . 

The final statement for this index may be the most 
interesting, as it taps the amount of additional time a teacher 
may devote ^^o school activities as a result of the merit schools 
program. In every district category, teachers feel they spend 
more time than is perceived by administrators. In Dade, a 
majority of teachers and administrators agree teachers spend more 
time (67% and 55%). This result is not surprising given that the 
school wide projects undertaken vould seem much more time 
consuming than fulfilling the optional criteria chosen by other 
districts. This conclusion is supported by the negative view of 
administrators in the salary supplement schools (54% disagreement 
versus 32% agreement) and the high level of undecided responses 
from their counterparts in the materials only districts (31% 
undecided) . Teachers, on the other hand, in both salary 
supplement and materials only districts, responded much more 
positively. 

District administrators generally support the view that 
instructional practices are improving because of the merit 
schools program. The percentage in agreement, though, is not in 
the majority (46%) and there is a significant percentage that 
disagrees (31%) (D-1-3) . 



INCREASED SCHOOL BASED MANAGEMENT PRACTICES 



Overwhelmingly, the results from all sources indicate that 
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the merit schools program produces increased teamwork among 
teachers, and among teachers and administrators, and has 
increased teacher involvement in decision making. In almost 
every case, principals and administrators are more in agreement 
than teachers. For those districts other than Dade that provide 
salary supplements, there is a greater amount of disagreement 
that these school based management results have occurred, with 
administrators disagreement more than teachers. 

In terms of responses to the statements that comprise the 
TEAMWRK index, the responses to the statement "teamwork among 
teachers is better", produced percentage's in agreement among all 
three district categories of teachers that ranged from 51-57, 
while for principals the range was 46-76 (A-l-f ;B-l-g) . 
Likewise, teachers agree that teamwork among teachers and 
administrators is better (48-55% in agreement) , while 
administrators agree to an even .greater extent (50-74%) (A-2-9;B- 
1-g) . In response to a series of statements concerning teacher 
involvement in decision making, a majority of teachers from both 
Dade County and the materials only districts agreed that teachers 
are more involved in choosing instructional methods (56% and 
59%) ; making decisions about the purchase of instructional 
resources and equipment (51% and 78%) ; school goals (65% and 
60%) ; and decisions about school activities and practices (54% 
and 53%) . Likewise, for school principals and administrators, 
the percentage of agreement for the same items produced an 
agreement range of 56% to 79% (A-3-a to e; B-3-a to e) . 

ERLC ^ 



60 

It is noteworthy that the educators from other salary 
supplement schools were more negative. For example, only 32% of 
teachers agreed that teachers are more involved in decisions 
concerning school policy and procedures (A-3-c) . Among 
administrators from these districts, the responses reflect a much 
smaller percentage of undecided opinion and a much higher 
percentage of disagreement. For the same statement, 40% of 
administrators disagreed that teachers help decide school policy 
and procedures, compared to 14% from Dade and 23% from materials 
only districts (B-3-c) . In terms of the purchase of 
instructional resources and equipment, the corresponding 
percentages of disagreement for administrators were 38% from 
salary supplement districts, 17% from Dade and 7% from materials 
only districts. 

In conclusion, there is a consistent majority agreement 
among teachers from the materials only districts to all four 
items. It seems likely that the merit schools program is 
achieving its goal of improving instructional practices in these 
districts. The Dade program reflects relatively positive 
responses as well. The picture in the other salary supplement 
districts, however, is unclear, as there is a much higher 
percentage of undecided responses along with a lower percentage 
of agreeme'^t. In addition, responses to the district 
administrator survey reflect a mixed result as the same number 
agree as do disagree (42%) that principals and teachers are 
interacting to a greater extent because of the merit schools 
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program. 

INCREASED ECONOMIC INCENTIVEvS FOR TEACHERS AND OTHER SCHOOL 
PERSONNEL 

There is a very clear connection between the distribution of 
Bier it school moneys and the positive or negative acceptance of 
the program by teachers and administrators. In those districts in 
which a higher percentage of schools are designated as 
meritorious, there is a greater acceptance of the merit schools 
program. As indicated by Table 8, for the most part, these 
districts distribute their funds prime rily for the purchase of 
instructional materials. There is much less of a sense of inter- 
school competition in these districts, because the merit school 
program is established so that any improvement in performance or 
even a mainten*^..ce of the current level of performance results in 
merit scho 1 designation. 

In those districts that provide primarily salary 
supplements, the program is viewed much more negatively. The ^e 
is a heightened sense ot competition among these schools, because 
a much siaaller number of schools ic likely to receive a merit 
school designation. There is also a greater likelihood that high 
stakes testing pressure is perceived In a negative fashion by 
teachers and principals in these schools. 

Another factor must be considered: the amount of salary 
supplement received by each teacher in a merit school. As 
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indicated by Table 12, Dade County teachers have aenerally 
received a much higher amount of a salary supplement than 
teachers in other salary supplement districts. Teachers in some 
of Dade schools may feel that the amount of extra effort required 
by the merit schools program may be worth the minimum of $500, 
with the possibility of receiving $1000, In districts where 
teachers received less than $200 a much more negative view of the 
program is common- 

COMPETITION AMONG SCHOOLS WITHIN EACH DISTRICT CATEGORY 

Most educators agree that the merit school program causes a 
sense of competition among schools. The perception that this 
competition exists, however, varies from district to district. 
Educators from the materials only districts reported less of an 
awareness of competition than the other two district categories, 
and were less likely to view competition in a positive sense. 

A majority of teachers and administrators from Dade and 
other salary supplement districts ag^se that the sense of 
competition exists (Dade: 57% and 65%; salary supplement: 75% and 
68%) . For the materials only districts, however, teachers report 
an agreement of 49%, while more administrators disagree than 
agree (48% to 41%) (A-8-a;B-8-a) . Similarly, a majority of Dade 
and salary supplement educators feel the district promotes this 
competition (53% to 63% range) , while only 32% to 44% agree and 
33% to 50% disagree from the materials only districts (A-8-b;B-8~ 




TABLE 12 

Dollar Amounts of Full Share per Teacher 
in Salary Supplement Districts 
(rounded to nearest dollar) 



YEAH 



County 


04/85 


85/86 


86/87 


87/88 


88/89 


Baker 


219 


90 


277 


155 


1090 


Bay 




800 


500 


150 


150 


Brevard 


992 


371 


330 


228 


166 


Calhoun (approx) 




400 


700 


600 


600 


Dade • 


1208 


637 


620 


570 


500 


E" 


2416 


1274 


1240 


1140 


1000 


Hardee (approx) 


1053 


405 




200 




Indian River 


450 


32S 


300 


270 


250 


Nassau 


543 


605 




369 


329 


Pasco 


403 


362 


26t 


226 


253 


Putnam (approx) 


700 


500 


400 


400 


300 


Sarasota 








100 


100 


Suinter 






895 


1829 


575 


Suwannee 


355 


313 


122 


129 




Data are based on 


information 


reported 


on the 


District 


Administra 



Questionnaire 
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b) . Unifonaally, less than a majority agree that students are 
aware of this competition (15% to 40%) , but only in materials 
only do a majority of educators disagree (57% to 64%) (A-S-cjB-S- 
d) . Again, an overwhelming majority of educators in Dade and the 
salary supplement districts agree that teachers are aware of the 
sense of competition (71% to 80%) while in the materials only 
districts a majority of teachers but a minority of administrators 
agree (55% and 40%) (A-S-djB-S-d) . 

A majority of teachers and administrators from all three 
district categories agree that the competition focuses on the 
standardized test scores (72%-89%) and thef other criteria (59%- 
71%) {A-8-f ,g7B-8-f ,g) ; but only a minority of teachers and all 
administrators except in Dade feel this competition is good 
(teachers: 30%-44% agree; administrators: 27% to 59% agree). 
Relating competition to increased student achievement is seen 
positively only in Dade (A-8-h;A-9-i;B-8-h, i) . In terms of 
increased cooperation among school employees, a majority of 
administrators agree that competition causes cooperation (54% to 
70%) , while only in Dade do a majority of teachers agree (A-9- 
k;B-9-k) . 

In summary, the sense of competition generated by the Merit 
Schools Program is viewed most positively in Dade County. For 
the materials only districts, competition is viewed as having the 
least amount of impact, and this impact is not seen as positive. 
In the salary supplement districts, competition is present, but 
is often seen more negatively than positively. 
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TEACHER TRANSFERS TO MERIT SCHOOLS FROM NON MERIT SCHOOLS 

There is little evidence that teachers transfer to schools 
designated as merit schools from schools that are not identified 
as meritorious. We found no agreement with this by any 
administrator interviewed. Principals and school administrators 
overwhelmingly disagreed with the statement that teachers tend to 
transfer to merit schools, with responses ranging from 55-76% (B- 
2-0) . Only in Dade, with a high of 24% in agreement, may there 
be some connection between the program and teacher transfers. 
District administrators strongly suppo^rted this evidence as 84% 
disagreed with a similar statement (D-1-9) . • 

These results are supported by other relevant facts. In 
those districts in which there are a high percentage of schools 
awarded merit school status, there would be no incentive for 
tra*isfers based on the program. In addition, teacher transfers 
occur for a multitude of reasons, with merit school status 
obviously one of the least important. 

CONCLUSIONS AND RECOMMENDATIONS 

Our analysis suggests that the merit schools program has had 
a positive impact in many respects. In general, teachers and 
administrators throughout Florida have a positive view of the 
program. Many educators also agree that the program provides a 
positive incentive for teachers to improve their instructional 
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practices* For the most part, districts are not biasing the 
awarding of merit school status away from those schools with a 
high percentage of lower level socioeconomic students. The 
program is decidedly helping schools to adopt school based 
management practices ♦ 

There are some improvements, however, that could bo 
implemented to help solve some of the weaknesses of the program. 
It is impossible to ascertain whether or not student learning is 
improving because of the program since the program requires the 
use of standardized achievement test scores to be interpreted by 
a statistical model that in many cases is not valid. The program 
could be altered to allow districts to adopt their own measures 
of student learning that would constitute alternatives tc 
standardized achievement tests. Florida Department of Education 
(DOE) staff could provide technical assistance to districts in 
this effort. 

We discovered that many district personnel had little 
knowledge of how the program was implemented in other districts. 
Many educators expressed misconceptions concerning program 
requirements- We strongly urge DOE staff to provide mechanisms 
by which educators from across the state can interact with each 
other and share information concerning what practices are most 
effective. 

The optional criteria need to be revisited as well. In some 
districts, the criteria were fixed by the district, and reflected 
activities that were relatively easy for teachers and 
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administrators to meet. In other districts, however, the 
requirement that improvement over a base line or last year's 
results be demonstrated was causing concern. Educators 
complained that since they were already operating at a high 
level, it would be very difficult to improve their scores on the 
required criteria. Although no cheating was mentioned during our 
interviews, documentation concerning the achievement of the 
optional criteria was not reviewed by district level personnel in 
almost all districts. 

To help solve these difficulties, several approaches can be 
suggested. First, districts could be encouraged to identify 
optional criteria that stimulate teachers and administrators to 
engage in activities that they would not otherwise have done. 
Second, more districts need to allow greater discretion to 
individual buildings in tenns of what specific optional criteria 
should be chosen. Third, districts should require the submission 
of documentation to a district level office, which in turn would 
submit a sample of this documentation to DOE. 

The program is not providing adequate economic incentive for 
teachers in most districts that provide salary supplements. In 
these districts as well, the sense of competition fostered by the 
program is viewed negatively. Also, in such districts, the 
percentage of schools identified as meritorious is likely to be 
relatively small (no more thsn 25-35%) . Unless the program can 
be funded at a high enough level so that every teacher receives a 
salary supplement amount that is viewed positively, districts 
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Should be encouraged to distribute the award money in ways other 
than salary supplements. The most positive view of the program 
is found in those districts where teachers choose to spend the 
money on equipment and materials only. For these districts as 
well, the sense of competition is the lowest, and the percentage 
of schools provided merit school status is the highest • A 
future, more effective merit schools program perhaps should 
eliminate salary supplements. 

The major benefit of the merit schools program seems to have 
little relevance to the legislative intention of improving 
instructional practices through providing economic incentives to 
teachers. The program permits schools to identify local needs 
and use the resources generated by the program to meet these 
needs. This benefit is most clearly seen in materials only 
districts* 
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APPENDIX A 



QUESTIONS USED AS A GUIDE FOR THE PERSONAL INTERVIEWS 
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i 1- 



SCHOOL ADHINISTRkTORS 

How has the QIIP program affected your school? 
(Can you tell me more about that?) 



? 2. How, if at all, has it affected the way you work with teachers? 



*3. How, if at all, has it affected the way teachers work? 



4. How, if at all, has it affected the way administrators from other buildings 
work together? 



5. How are the school's QIIP goals determined? 



6. In this building word term is used for the QIIP or Merit Schools Program? 



If classroom teachers were asked to play a word association game by listing 
words that first come to mind, what would they say after hearing ? 



7- Some people say that schools are doing a better job because test scores are 
getting higher. Other people say that as test scores go up, real learning 
declines. 

Why do you think one point of view is more correct than the other? 



8. What happens to the QIIP prog. dm funds? 



(how much, all state funds?, any local funds?) 

9. If- you could make major changes in the QIIP program , what changes would you 
make? 



; 10. Title of position 



Years: at this school 



; in the county 



11. What else is important to know in order to understand the process or 
impact of the QIIP program in your county? 
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CLASSROOM TEACHER 



1. How has the QlIP program affected your school? 
(Can you tell me more about that?) 



2. How 



, if at all, has it affected the way you work with other teachers? 



3. How, if at all, has it affected the .ay administrators work with teachers? 

4. What kinds of things have happened to make you think that standardized test 
scores are important? 



5. in what ways, if at all, have the tests Influenced what content yo. teach 
and how you teach? 



6. How are 



the school's QIIP goals determined? 



7. in this building word term is used for the QIIP or Merit Schools Program? 

if'classroom teachers were asked to play a word association game by listing 
LrdstJat first come to mind, what would they say after hearing 



better job because test scores are 
test scores go up, real J 

Z'T,on think one point of vie. is n,ore correct than the otherT 



a. So„e pe. .e sa, that schools are doin, ^^^^J^-J-^.f^r^;, I-^i^, 
getting higher. Other people say that as test scores go y. 



9. What happens to the SUP program funds? 

10. If you c^-uld make major ch-^ng.s in the QIIP program , what changes would 
you mcike? 



11. Title of position .; 

Years: at this school ' ^" 



U. What else is important to know in order to understand the process or 



12. What else is impoi i-a'^- ^.^ ^„„„.„, 
, rn^^-L-npact of the QIIP program in your county. 
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MAIL QUESTIONNAIRES 
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QU E 3 T I ONN A X RE 

During interviews, school administrators and classroom teachers have 
reported many opinions about changes caused by the QUIIP or Merit Schools 
Program. They have reported positive changes, negative changes, no change 
at all, and many opinions about topics related to the process of identifying 
and awarding merit schools. 

Directions t Based on your personal experience , indicate the extent to which 
you agree~ or disagree with each of the following statements. Fill m the 
space that goes with your answer. 

1 = strongly disagree 

2 = disagree 

3 = undecided 

4 = aaree STRONGLY STRONGLY 

5 = strongly agree DISAGREE = 1 to 5 = ACREE 

1. Because of the merit schools program, principals 

and teachers are interacting to a greater extent on , , . , , > , an f 

a wide range of decisions. (1) ^2) (3) I4j i^; 

2. The merit schools program has added little to the 
to the activities and behaviors that teachers and 

principals typically perform in our district. (1) U) u; i = ^ 

3. The merit schools program is achieving its primary 

goal of improving instructional practices in our ,^^ cj^ i&) is) 

district. ^ ' ^ ' ^ ' 

4. Overall, there is a positive feeling among roost 

teachers toward the Merit Schools Program. (D U) l^; 

5. Because of the pressure caused by the consequences 
of the standardized test scores that are part of the 
Merit Schools Program, inappropriate procedures methods 

are used by teachers and principals. U) u; 1**^ 

6. Our district's merit school program is working 

very well. ^ ' ^ ' 

7. District testing experts believe the statistical 
model that interprets the standardized test pcxxc of 

the merit schools criteria is a valid model. U; u; v*' 

8. There have been few problems with implementing the 

merit schools program. ^''^ ^ ' 

9. Transfers by teachers to merit schools have 

increased since the merit school program has begun. ID U) i^i id; 

10. The merit schools program: 

a. should be dropped \]\ 2 3 4 b 

b. should be kept, if modified U 2 3 4 5 

c. should continue in its present form. U) U) 1**^ v^' 



1 over ) 



In item 11 below, there are five statements concerning the benefits of the ; 
Merit schools Program. Please rank these five, marking (1) as the most 
important, (2) as the next most important, ecc. 

11. Rank order the benefits of the Merit Schools Program. 

a. The additional dollars it provides to teachers 
and/or schools 

b. The increased interaction between teachers and 
principals 

c. The improvement in standardized test scores 
ri. The increased community involvement 

d. The increased school pride 

12. If any schools in your district have chosen not to participate in the 
program, why do you believe they have not participated? 



(1) (2) (3) (4) (5) 

(1) (2) (3) (4)* (5) 

(1) (2) (3) (4) (5) 

(1) (2) (3) (4) (5) 

(1) (2) (3) (4) (5) 



13. If the district uses any methods to audit each school's report of 
achievement on the optional criteria, please briefly describe the aud 
procedure. 



14. Please complete the following table. If a complete set of records is not , 
available, please provide your best es*-imates. I 

84/85 85/86 86/87 87/88 88/89^ 

a. Number of schools participating in the ^ 
merit schools program. 

b. Number of schools who qualified as meritorious 

(if appropriate, indicate by level of merit) . J \ 

c. Number of schools that failed to meet : 

standardized test criteria 

optional criteria 
both criteria 

d. The dollar amount of a full share per teacher 



THANK YOU FOR YOUR HELP IN ANSWERING SOME IMPORTANT QUESTIONS ABOUT 

SCHOOL ADMINISTRATORS' OPINIONS 

The State's code number for your school district is . 

OTHER COMMENTS: 
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During interviews, school administrators have reported many opinions about 
-hanges caused by the QUIIP or Merit Schools Program. They have reported 
■positive changes, negative changes, no change at all, and many opinions 

about topics related to the process of identifying .and awarding merit 

schools • 

Directions ; Based on your personal experience , indicate the extent to which 
you agree "or disagree with each of the following statements. Fill m tne 
space that goes with your answer. 



1 = strongly disagree 

2 = disagree 

3 = undecided 

4 = agree 

5 = strongly agree 



STRONGLY STRONGLY 
DISAGREE = 1 to 5 = AGREE 



I BECAUSE OF THE MERIT SCHOOLS (QUIIP) PROGRAM: 



I. SCHOOL CONDITIONS 

a) Student learning has improved. 

b) Teachers devote more hours working for the school. 

c) More student time is used for learning activities. 

d) Teachers use more effective instructional methods. 

e) Student attendance is better. 

f ) Teamwork among teachers is better. 

g) Teamwork among teachers and administrators is better. 

h) The instructional materials or equipment is better. 

i) Students take standardized tests more seriously, 
j) More time is used to teach test taking skills. 

k) Teachers require student participation in activities 

related to merit school goals. 
1) Parent participation in school activities is better, 
m) Community pride in the school is better, 
n) Communication among administrations from different 

schools is better. 

0) Teachers tend to transfer from other schools to 
schools that o^ten earn merit school awards. 



II. TEACHERS AIIE MORE INVOLVED IN DECISIONS ABOUT: 

a) instructional mathods 

b) purchase of instructional resources and equipment 

c) school policy and procedures 

d) school goals 

e) other grade level or school activities and practices. 



{ 1) 


(2) 


13) 


(4) 


{5] 


(1) 


(2) 


(3) 


(4) 


15] 


ID 


I ^ J 








(1) 


(2) 


'3) 


(4) 


{5: 


(1) 


(2) 


(3) 


(4) 




(1) 


(2) 


(3) 


(4) 


(5 


(1) 


(2) 


(3) 


(4) 


(5' 


(1) 


(2) 


(3) 


(4) 


(5^ 


(1) 


(2) 


(3) 


(4) 


(5 


(1) 


(2) 


(3) 


(4) 


(5 


(1) 


(2) 


(3) 


(4) 


(5 


(1) 


(2) 


(3) 


(4) 


(5' 


11) 


(2) 


(3) 


(4) 


(5 


(1) 


(2) 


(3) 


(4) 


(5 


(1) 


(2) 


(3) 


(4) 


(5 


(1) 


(2) 


(3) 


(4) 


(5 


(1) 


(2) 


(3) 


(4) 


(5 


(1) 


(2) 


(3) 


(4) 


(5 


11) 


(2) 


(3) 


(4) 


(5 


(1) 


(2) 


(3) 


(4) 


{5 
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STRONGLY 
DISAGREE = 1 
(1) (2) 



STRONGLY 
to 5 = AGREE 
(3) (4) 



(1) 
(1) 
(1) 

(1) 
(1) 



(2) 
(2) 
(2) 

(2) 
(2) 



(3) 
(3) 
(3) 



(4) 
(4) 
(4) 



(3) (4) 
(3) (4) 



(1) (2) (3) (4) 
(1) (2) (3) (4) 



(1) (2) 
(1) (2) 
(1) (2) 



(1) 
(1) 
(1) 
(1) 



(2) 
(2) 
(2) 
(2) 



(1) (2) 



(3) 
(3) 
(3) 

(3) 
(3) 
(3) 
(3) 
(3) 



(4) 
(4) 
(4) 

(4) 
(4) 
(4) 
(4) 
(4) 



(5 
(5 
(5 
(5 

(5 
(5 



(1) (2) (3) (4) (5 



f:S|xpress your personal opinion about the Merit Schools Program by indicating 
ithe extent to which you agree or disagree with the following statements. 

III. THE MERIT SCHOOLS (QUIIP) PROGRAM: 

1^) is fair to all schools. 

lib) has a positive impact on my school. 

§c) provides educators a reward for good work. 

IS) has pushed schooling toward trivial learning. 

I|e) promotes practices that diminish the quality of 
%T education while test scores increase. 

lif ) is very easy, requiring no extra effort. 

:%) rewards the most improved rather than the highest 
quality schools . 

yti) uses test scores that give false information 
about student learning. 

^A) uses other school criteria that are important. 

allows politics to get in the way of true teaching 
quality awards. 
|:k) has resulted in wise use cf state funds. 
fl) is a credible program for school improvement. 

m) requires an administrator to devote more time to 

leading school activities. 
tn) results in competition that has corrupted schools. 
^:0) should be dropped altogether, 
gp) should be continued, only if modified. 
I?q) should be continued as is. 



(5 
(5 

(5 
(5 
(5 

{5 
(5 
(5 
(5 
(5 



•a) 



Id) 



IV. Background Information 

What term best describes your school: 

Elementary (1) 
Middle School/ Junior High {2) 
High School (3) 
How many years of school administrative experience do you have? 
To your knowledge, for how many years has your school 

participated i.\ the Merit Schools Program? 
How many times has your school been a Merit (QUIIP) School? 
How many years have you been an administrator at your 
current school? 



THANK YOU FOR YOUK HELP IN ANSWERING SOME IMPORTANT QUESTIONS ABOUT 

SCHOOL ADMINISTRATORS' OPINIONS 
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r The State's ccie number for your school district is 
IcOMMENTS : 



CLASSROOM TEACHER QUESTI OlSTNAI RE 

(form A) 



During interviews, classroom teachers have reported many opinions about the 
changes caused by the QUIIP or Merit Schools Program. Teachers have 
reported positive changes, negative changes, no change at all, and many 
opinions about topics related to the process of identifying and awirding 
merit schools. 

Directions : Based on your personal experience ^ indicate the extent to which 
you agree or disagree with each of the following statements. Fill in the 
space that goes with your answer. 

1 = strongly disagree 

2 = disagree 

3 - undecided 

4 = agree STRONGLY STRONGLY 

5 = strongly agree DISAGREE - 1 to 5 = AGREE 



BECAUSE OF THE MERIT SCHOOLS (QUIIP) PROGRAM: 



I. School Conditions 



a) 


Student learning has improved. 


(1) 


(2) 


(3) 


(4) 


(5 


b) 


Teachers devote more hoxirs working for the school. 


(1) 


(2) 


(3) 


(4) 


(5 


c) 


More student time is used for learning activities. 


(1) 


(2) 


(3) 


(4) 


{5 


d) 


Teachers use more effective instructional methods. 


(1) 


(2) 


(3) 


(4) 


(5 


e ) 


Student attendance is better. 


(1) 


(2) 


(3) 


(4) 


(5 


^ \ 
J* / 


Teamwork among teachers is better. 


(1) 


(2) 


(3) 


(4) 


(5 


g) 


Teamwork among teachers and administrators is better. 


(1) 


(2) 


(3) 


(4) 


(5 


h) 


The instructional materials or equipment is better. 


(1) 


(2) 


(3) 


(4) 


(5 


x) 


Students take standardized tests more seriously. 


(1) 


(2) 


(3) 


(4) 


(5 


j ) 


More time is used to teach test taking skills. 


(1) 


(2) 


(3) 


(4) 


(5 


k) 


Teachers require student participation in activities 






(3) 


(4) 


(5 




related to merit school goals. 


(1) 


(2) 


1) 


Parent participation in school activities is better. 


(1) 


(2) 


(3) 


(4) 


(5 


in) 


Community pride in the school is better. 


(1) 


(2) 


(3) 


(4) 


(5 




II. TEACHERS ARE MORE INVOLVED IN DECISIONS ABOUT: 












a) 


instructional methods 


(1) 


(2) 


(3) 


(4) 


(5 


b) 


purchase of instructional resources and equipment 


(1) 


(2) 


(3) 


(4) 


(5 


c ) 


school policy and procedures 


(1) 


(2) 


(3) 


(4) 


{5 


d) 


school goals 


(1) 


(2) 


(3) 


(4) 


(5 


e) 


other grade level or school activities and practices. 


(1) 


(2) 


(3) 


(4) 


(5 



.^.Express your personal opinion about the Merit Schools Program by indicating 
i^^the extent to which agree or disagree with the following statements. 



III. THE MERIT SCHOOLS (QUI IP) PROGRAM; 



STRONGLY STRONGLY 
DISAGREE = 1 to 5 = AGREE 
(1) (2) (3) (4) (5; 

(2) (3) 

(2) (3) 

(2) (3) 



(1) 
(1) 
U) 



(4) 
(4) 
(4) 



(5 
(5 
(5 



|:a) is fair to all schools. 
Mb) has a positive impact on my school. 
?vc) provides teachers a reward for good work, 
-fjd) has pushed schooling toward trivial learning. 
f/e) promotes practices that diminish the quality of 

V education while test scores increase. (1) (2) (3) (4) {5 

|:f) is very easy, requiring no extra effort. (1) (2) (3) (4) (5 

/g) rewards the most improved lather than the highest 

jV quality schools. (1) (2) (3) (4) (5 

^ii) uses test scores that give false information 

?: about student learning. (1) (2) (3) (4) (5 

fi) uses other school criteria that are important. (1) (2; (3) (4) (5 

1^ j) allows politics to get in the way of true teaching 

quality awards. (1) (2) (3) (4) (5 

1%) has resulted in wise use of state funds. (1) (2) (3) (4) (5 

ID is a credible program for school improvement. (1) (2) (3) (4) (5 

l^o) results in competition that has corrupted schools. (1) (2) (3) (4) (5 

|ip) should be dropped altogether. (1) (2) (3) (4) (5 

q) should be continued, if modified. (1) (2) (3) (4) (5 

pt) should be continued as is. (1) (2) (3) (4) (5 



a) 



» 



IV. Background Information 

What term best describes your school: 

Elementary (1) 
Middle School/ Junior High (2) 
High School (3) 
How many years of full time teaching experience do you have? 
To your knowledge, for how many years has your school 

participated in the Merit Schools Program? 
How many times has your school been a Merit (QUIIP) School? 
How many years have you taught at your current school? 



THANK YOU FOR YOUR HELP IN ANSWERING SOME IMPORTANT QUESTIONS ABOUT 

TEACHERS' OPINIONS 



IP 



% The state's code number for your school district is 

iScOMMENTS: 



AS SROOM ACHEI?. QUESTX ONIST AI RE 

(form 1.) 

f.^During interviews, classroom teachers have reported many opinions about the 
Pchanges caused by the QUIIP or Merit Schools Program. Teachers have 
^.reported positive changes, negative changes, no change at all, and many 
If .opinions about topics related to the process of identifying and awarding 
|5- merit schools. 

i" Direct ions : Based on your personal experience , indicate the extent to which 
I; you agree or disagree with each of the following statements. Fill in the 
space that goes with your answer. 

1 = strongly disagree 

2 = disagree 

3 = undecided 

4 = agree STRONGLY STRONGLY 
^: 5 = strongly agree DISAGREE = 1 to 5 = AGREE 

I. THE MERIT SCHOOLS (QUIIP) PROGRAM: 

is fair to all schools. 

has a positive impact on my school. 

provides teachers a reward for good work. 

has pushed schooling toward trivial learning. 

promotes practices that diminish the quality of 
education while test scores increase. 

is very easy, requiring -no extra effort. 

rewards the most improved rather than the highest 
quality schools. 

uses test scores that give false information 
about student learning. 

uses other school criteria that are important. 

allows politics to get in the way of true teaching 
quality awards. 

has resulted in wise use of state funds. 

is a credible program for school improvement. 

results in competition that has corrupted schools. 

should be dropped altogether. 

should be continued, if modified. 

should be continued as is. 



1 1 ) 


f 2 ) 


( 3 ) 


(4) 


(5) 


1 1 ) 
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(4) 


(5) 


1 1 ^ 
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\ *t ) 
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(4) 
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(5) 
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(4) 


(5) 




(2) 
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(4) 


(5) 
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(3) 


(4) 


(5) 




(2) 


{3) 


(4) 


(5) 




(2) 


(3) 


(4) 


(5) 
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(3) 


(4) 


(5^ 
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(3) 


(4) 


(5) 
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(3) 


(4) 


(5) 
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(3) 


(4) 


(5) 




(2) 


(3) 


(4) 


(5) 




(2) 


(3) 


(4) 


(5) 
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II, STANDARDIZED TESTS 



^Standardized tests such as CTBS, CAT, CAP, Iowa, etc. are given and used 
|£n a variety of ways within different schools. To what extent do you 
fagree that the following statements describe practices in your school? 



IN MY SCHOOL: 



STRONGLY 
DISAGREE 

fa) the curricula are designed to teach the content on 
'■'^ the test in order to match what is taught and tested 
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ib) teachers are prohibited from reviewing test content. 

items from the tests are used (without modification) 
on other tests or student exercises. 

B) items from the tests are slightly modified, then used 
on other tests or student exercises. 

■;4) items on the tests are used as models for item format 
when writing other tests or student exercises. 

^) experienced teachers remember items from the test and 
focus part of their instruction on those items. 

fcf) test administration procedures are followed exactly 
as printed. 



STRONGLY 
: 1 to 5 = AGREE 

(1) (2) (3) (4) (5) 
(1) (2) (3) (4) (5) 

(1) (2) (3) (4) (5) 

(1) (2) (3) (4) (5) 

(1) (2) (3) (4) (5) 

(1) (2) (3) (4) (5) 

(1) (2) (3) (4) (5) 



la) 



I*) 
|) 

%) 
%) 



IV. Background Information 

What term best describes your school: 

Elementary (1) 
Middle School/ Junior High (2) 
High School (3) 
How many years of full time teaching experience do you have? 
To your knowledge, for how many years has your school 

participated in the Merit Schools Program? 
How many times has your school been a Merit (QUIIP) School? 
How many years have you taught at your current school? 



THANK YOU FOR YOUR HELP IN ANSWERING SOME IMPORTANT QUESTIONS ABOUT 

TEACHERS' OPINIONS 



The State's code number for your school district is 
ifcOMMENTS : 
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35-6 % 
b 

38-41 .3 

■M 



I 

If 



STATE ASSESSMENT OF* TME 

QXJA31.XT'J£' INSTRXJCTXON 
XNCENTIVES E>ROGRAM (QTJXXE*) 
oommonly Icnown as tzlrxe 
MERXT SCHOOXiS E>R.OG5?.A3yi 

SXJE»S>Xi,EMENT TO QXJESTX02SINAXRE 



please complete this supplement to the questionnaire. After you have 
Answered the items, insert the supplement between the two middle pages. 
Ichen fold and return the completed questionnaire and supplement. 

locus of additional concern: Does the Merit Schools Program generate 
^com-petition among schools and what are the results? 

I — 

|- 

l)irections: Based on your personal experience , indicate the extent to 
*which you agree or disagree with each of the following statements. 
[Fill in the space that goes with your answer. 



1 = strongly disagree 
? 2 = disagree 

3 = undecided „««,«M/^Tt» 

4 = agree STRONGLY STRONGLY 
I 5 = strongly agree DISAGREE = 1 to 5 = AGREE 

Because of the merit schools (ouiip) program 

|) There is a sense of competition among schools. (1) (2) (3) (4) (5) 

3.) The school district promotes a sense of r- wow^^ f«^^ 

I competition among schools. '^^f 
i) Students are aware of a sense of competition. (1) (2) (3) (4) (5) 

i) Teachers are aware of a sense of competition. {">) (2) (3) (4) (5) 

i) Merit School competition among schools is good. (1) (2) (3) (4) (5) 

Sx)Pl If you answered answered (1), (2), or (3) to all of the items above, 
do not continue. If you answered agree (4) or strongly agree (5) to 
one or more of the itass above, please continue. 

f) Competition focuses on standardized test scores. 
3) Competition focuses on other Merit School criteria, 
a) Competition among schools helps to improve student 
scores on standardized achievement tests. 

|) Competition among schools helps to improve student 
I academic achievement in general. 

I) Competition among schools helps to improve student 
achievement of other important school objectives. 

Competition among schools helps to improve 
a sense of team cooperation among school 
employees ( administrators, teachers, staff) 

I'd 

t'. o 

i ERIC THANK YOU FOR YOUR EXTRA HELP WITH THIS STUDY 

§0. 103 
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(1) 
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(2) 
(2) 


(3) 
(3) 


(4) 
(4) 


(5) 
(5) 


(1) 


(2) 


(3) 


(4) 


(5) 
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(5) 
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(3) 


(4) 


(5) 
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(2) 


(3) 


(4) 


(5) 
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'0 MERIT SCHOOLS 

|: STAKEHOLDERS CONFERENCE 

>^ 
I- 

I^Tipril 26, 1990 



^Dear Stakeholder 



ft W6 have enclosed two types of information that you should review prior to 
|*he^confI?ence!^ Se fSst is a document that surmarizes state^r^^^^^ 

I taSuiSoSs represei? ?he results obtained from four survey instruments: 
i two i^truments mailed to a sample of teachers one ^^f trument mailed^t^ . 
^. sample of school administrators, and one instrument mailed to the schoox 
i district coordinator in each of the participating districts. .|| 

I The two instruments mailed to teachers and the instrument mailed to sg^^ 
t administraors contained many of the same items. The strategy permittea a^^^^ 
^comparison between responses on different instruments. 

? One of the enclosed tabulations compares ^^^sponses of teachers and^^^ J 
S^SlSfofSpSri^fresponsIs Of school aa^ini^ 



§1' 

m 



that use three different strategies for merit school awards. 

The tabulations of survey responses from teachers J^d administrators who 
work in different levels of schools (elementary, "«iddle/jr. Jjf ' 
schools) will be available during the conference. Overall, the responses ^ 
were very similar by level of school. 

t yoS^cL£ler'?o SfLportant. Wer a few issues that seem to be of ^ 
current interest to some legislators include the following. 

Interpretations : 

1. Is there evidence that the program has produced school 
improvement? 



I? 
f 



2. Is there evidence that teachers have become more 
involved with school based management? 




Does the amount of a salary supplement award have an impact J: 
on the effectiveness of the program? 



Do salary supplerafent awards have a larger or smaller influence on 
?eachels^ef fS?ts to improve schooling than school awards used for 
the.,raaterials_a^^^^^ . 



5. Does the program stimulate school improvement in districts that 
award a high percentage of their schools? 

6. Does the program work equally well at all school levels: 
elementary, middle/ jr. high, high school? 

7. Are standardized test scores a meaningful indicator of 
school effectiveness? 

8. Have school districts created adequate models for identifying 
schools that produce students who score above expected levels on 
standardized achievement tests? 



Potential actions: 
9. 



10. 



Eliminate improved standardized achievement test scores as 
a criterion in the Merit School Program. 

Require school districts to develop alternative 
indicators of improved school effectiveness. 



11. Eliminate high schools from the Merit Schools program. 

12. Require districts that provide salary supplements to 
give awards of at least $500, or some amount, even if it 
would limit the number of schools selected. 

13. Charge the DOE with the resi.onsibility of facilitating 
exchanges between districts related to indicators of 
effectiveness and strategies for achieving merit: school goals. 

14. Charge the DOE with the responsibility to monitor and 
approve strategies for identifying schools that are 
identified as merit schools. 

we know that you will identify other issues that need to be discussed. 

Thank you for taking the time to attend the stakeholders conference. We 
look forward to working with you. 





Robert R. Lange S^S?- ^^S:^ in «^ration 

■ Educational Foundations , Public Administration 

rsity of Central Florida University of Central Florida 

. ERJC 



■FX.ORXDA • S MERIT SCMOOX-S 



HOW XT FXJlSICTIOlSr AND 

WHAT IS ITS I M]? ACT 7 



STAKE HOLDERS CONFERENCE THURSDAY & FRIDAY 

MAY 10-11, 1990 
Orlando, Florida 

The University of Central Florida, 

CEBA II Building 
College of Business Administration 
Second Floor Room 209 

Purposes: * review preliminary findings of a state funded 

study of the Florida Quality Instruction 
Incentives Program 

• assist in the interpretation of the findings 

* participate in identifying recommendations for 

DOE and legislative consideration 



Invited Participants: School District Personnel 

program coordinators 
directors of testing 
principals 
teachers 

teacher union representatives 
Department of Education Personnel 
State Teacher Union Representatives 
Legislative Staff Members 
State Legislators 




TRAVEL AND PER DIEM MUST BE PROVIDED BY PARTICIPANTS 
CONFERENCE FORMAT AND SCHEDULE 

THUP^DAY MAY 10 CEBA II Building — uol. of Business Admin. 

2nd Floor 
Room 209 

University of Central Florida 
Orlando, Florida 



Sign In 

Introductions and Project Overview 
Assignment to Work Groups 

Review of data or other information and 
prepare group prelijninary reactions 

Identify special issues based on local 
experiences and/ or study data 

Presentations of group preliminary 
reactions and special issues 



• Assignment to New Work Groups 

• Share examples of success and concerns 

related to specific issues 

• Formulate recommendations for refinement 

of the program 

Break 

• Presentations of group recommendations 

and general discussion 
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12:30 p.m. - 1:00 
1:00 p.m.- 1:30 

1:30 p.m.- 3:30 
3:30 p.m.- 4:30 



FRIDAY MAY 11 
8:15 a.m.- 8:30 
8:30 a.m.- 10:15 

10:15 a.m.- 10:30 
10:30 a.m.- 11:30 




APPENDIX D 



STAKEHOLDERS' CONFERENCE PARTICIPANTS 
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MERIT SCHOOLS PROGRMT 
STAKEHOLDERS* CONFERENCE PARTICIPANTS 
Kay 10-11, 1990 



F. Coordinators 
Wendell Lawther 
Bob Lange 
Diane Winston 

D. 0. E* Representatives 
Rufus Ellis 
Brian C\irry 

prevard ountv 
Sharon Vaughn 
Joy Salamone 

Marion County 

Jim Noell 

Mary Ann DeLong 

Nassau County 
Carolyn Parks 
Mildred Campbell 

Orange County 
Lee Baldwin 
John Hawco 

Pasco County 
Myndall Stanfill 
Jim Davis 
Steve Dubendorfer 

Pinellas County 
Stisve lachini 
Kate Herrington 
Brenda Leasure 
Frank Garcia 

St. Lucie County 
Stephen Bouzianis 
Darlene Dawson 

Seminole County 
Ernest Cowley 
Ted Barker 
Jim Dawson 
Jim Elliott 
Jim Neville 



APPENDIX E 



RESPONSE TO TKE REPORT ENTITLED: FLORIDA'S QUALITY INSTRUCTION 
INCENTIVES PROGRAM: ALLOCATION PATTERNS OF MERIT SCHOOLS 1984-1989 



MEMORANDUM 



TO: 



Bryan Curry 



rrom: 



Bob Lange and Wendell Lawther 
UCF 



Subject 
Date: 



Recciranendations by Barbara White 
April 12, 1990 



Dear Brian; 

In accordance with your request, the following is our 
•review of the recommendations found in in the report 
entitled: Florida* s Quality Instruction Incentives Program: 
Allocation Patterns of Merit Schools - 1984-1989. We are 
basing our comments on: 

1. review and analysis of district plans 

2. review and analysis of records of award patterns 

3. interviews with school administrators and teachers in 
a purposive sample of schools 

4. a preliminary analysis of survey data obtained as part 
of our study of the Florida QUIIP or Merit Schools 
Program. 



As implied in Barbara White's report, her 
recommendations are based on prior value orientations and a 
logical contingency analysis of district plans and records 
of awards. She had no information about what is happening 
in schools because of the program. 

The recommendations reflect an accurate review of the 
plans submitted by each district. However, that analysis is 
too limited. There is no demonstrated congruency between a 
logical analysis of the plans and the Liipact of the Merit 
Schools PrograiTt on schools. 

We believe that some of the assvimptions implicit in the 
report have merit and others have not been demonstrated to 
be valid. Our data will challenge the beliefs that underlie 
some of the reccmroendations. 

The following items examine each of the specific report 
recommendations in more detail. 

(Item numbers correspond to recoiranendation numbers on page 
18 of Barbara White's report.) 



1. Name Change , 
vniether or not the name of the program should be changed is 
a value orientation. In most districts, the focus has long 
been on awarding improved schools and providing better 



instructional materials and equipment to facilitate better 
student learning. The recommendation is a moot point, as 
our analysis shows that the focus has already changed. 

2. Amount of salary awards 

There is no empirical data to support the belief that the 
size of salary supplements results in improved teaching and 
learning. Perceptions of the impact of the program on the 
quality of teaching and learning is much more positive in 
those districts where salary supplements are not awarded 
than it is in districts that award salary supplements. In 
Dade county, the only large county in which all of the funds 
are given in salary awards and in which t:he awards are the 
largest, teachers and administrators report a lower level of 
impact than in counties that use the funds for materials and 
equipment . 

Our analysis is incomplete, it may be that the existence of 
salary supplements causes competition that has provided a 
negative perception of the program for many teachers. 

3. Criteria 

We are concerned with the basis for this recommendation. It 
assumes that there is no connection between the optional 
criteria that each school chooses and improved learning. 
This assumption may not be correct. There are no data in the 
state's records or in our study to address this issue. 

Based on a review of current literature, and not on any 
results of our study, we agree that the law should be 
changed to allow each district to develop alternatives to 
multiple choice tests. The validity of standardized test 
scores derived from old norms is very questionable. The 
"High Stakes Testing" literature would imply that programs 
that reward teachers and schools based on test scores result 
in practices that invalidate the scores. We have some 
indications of that result, but our data is not directly 
related to the issue. 



4. DOE authority 

This recommendations is a state procedural issue for which 
there are no real data other than a report of how the DOE 
currently operates. As general opinion, we agree with the 
recommendation and would support a more viable role for the 
Florida DOE in the monitoring of the Merit Schools Program. 
Because most districts act entirely independently and have 
no idea of what other districts are doing and what seems to 
work well in other districts, this intend should include a 
cross district exchange and sharing of results. 

If this recommendation is to be carried out, though, the 
legislature must recognize that more funds and staff support 
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must be provided to the DOE. Some legislators reject that 
idea because they believe that DOE staffing is in "fat city" 
and could do much more than they currently do without any 
increase in funding. Do you have data on that issue? 

5. Learning criteria 

The recommendation assumes that learning and attitude are 
independent. There are no data to support that position. 
We did not study that issue. 

We agree with the implication that for some school districts 
the optional criteria are weak and set up so that it is 
almost impossible for the schools not to achieve them. 
However, the impact of the optional criteria is often much 
greater that meets the eye. Our interview data are 
testimonial and based on multiple indicators rather than on 
single rimple indicators. 

The use of optional criteria is extremely varied by district 
and by school within district. There is little or no 
monitoring of the achievement of the criteria. 
There needs to be a clearer analysis of the impact that the 
achievement of the optional criteria are having on the 
primary goals of improved teaching and learning. There is a 
need to broaden the range of optional criteria available to 
each school. 

Each building could devise a plan outlining its needs, how 
it would propose to meet those needs, and how the meeting of 
those needs could be evaluated. 

We agree that school improvement standards (not 
participation standards) should be measured before and after 
implementation. Our study shows that this is occurring in 
many districts already. 

This area of work is in great need of development as a 
strategy for obtaining the results intended by the 
legislature. It is our impression that people who do not 
work in schools on a regular basis have no understanding of 
the importance and impact of many of the optional goals used 
by dome districts. 



6. Use of standardized tests 

We agree with this recommendation because there is no 
generally accepted way to use the results of standardized 
tests for the purposes outlined in the current law. We 
believe that most of the models currently in use by school 
districts to select merit schools do not achieve their 
intended purposes. A mathematical study of this issue is 
outside the scope of our project and would incl\xde an 
analysis of the validity of test scores based on old norms. 
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adjustments of scores in schools that have long produced 
students who achieve above expected levels, adjustments of 
scores in schools that have high percentages of students who 
change from year to year, any many related factors. 

We did find, though, that school administrators were not 
unhappy with this requirement, primarily because they were 
not in a position to provide alternatives to the use of 
standardized tests other than using the optional criteria. 
They have grown to live with the use of standardized test 
scores regardless of their belief in their value and 
meaning. 

7. Eliminate High Schools 

We do not support this recommendation. Our data show that 
there is little difference in the perceived impact of the 
Merit Schools program across school level; elementary, 
middle/ jr. high, high school. 

We feel that making major changes in the program at this 
time may be premature. There needs to be a greater effort 
to involve school districts who have been operating the 
program to come together and analyze the strengths and 
weaknesses of what they are doing. 

If the merit schools program is to be replaced with a 
program similar to accountability grc.its, the data suggest 
that it is just as appropriate at all school levels as it is 
at the high school level. Perhaps that would be "tossing 
out the baby with the bath." In general, there is a 
positive reaction to the merit schools program. Why not 
work to improve it rather than replace it with another 
program that would take several years to grow and become 
effective? 

As long as we are dealing with recommendations based on 
value orientations, consider the following: 
The state legislature should require greater formative 
evaluation and imporvement in the way that districts 
implement the merit schools program. The program should 
vary by district. The legislature needs to learn that 
programs with potential must grow into effectiveness over 
time, 5 to 10 years or more, and that their constant change 
of program mandates results in a lowering of school 
effectiveness and a destruction of the quality of schooling. 
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EXAMPLES OF STATISTICAL MODELS 
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Attachment E 



EXAMPLES OF STATISTICAL MODELS 
FOR USE !N SELECTING MERITORIOUS SCHOOLS 



The following statistical models are provided as illustrations of ways a district might meet the intent 
of Section 231. 532l3)(n.F S. Student achievement on verbal and ;uantiutive achievement tests is a 
required component of a district plan, with the additional requirement of at least one of the standards 
in subsections (3)(f)4.a. through (3)(n4.d. Tb^an may also include other district sUndards which go 
beyond these minimum requirements. Studjent achievement on verbal and quantitative tests may be 
combined into a composite score for determining" the^^schools which will be designated ^'meritorious. 
Refer to Scenarios A and B on pages E-2 and E-3 for examples of how this might be done. 

The examples included here mav be used as models and may also serve to stimulate the design or 
development of other models that will meet the intent of subsection (3)(n. To esublish realisUc goals, 
districts are urged to examine students' previous test performance (from year to year) pnor to 
selecting a method for weighting or combining achievement with other goal categories of the plan 
Further, the numerical values used in the following examples are for illustration purposes only and 
may not be suiuble for your district 

The 1986 Legislature deleted from the statute any reference to an •^upper quartile** of 
schools* or to the "too few schools" model, which were part of the program during 1984 and 
1985. Therefore, districts are free to design plans which result in the designation of any 
number of schools as "meritorious" as long as the basic criteria previously described are 
meu If the district chooses to use statistical procedures to limit the number of meritorious 
schools to the "upper quartile" this is permissible. However, it is not required. For this 
reason, the illustrations which follow do not make a distinction between the upper 
quartUe- and *'too few schools" models which were used in previous y ears. 

Method I 

Use of a school-anticipated or predicted mean score obtaine d from the administration of a 
nationaMv>normed test with school-onented abilit v as a predictor Students are given an 
i[bII7t7tes; pnor to taking the achievement battery. The predicted score for an mdividual 
student usuallv represents the achievement score obtained by other students of similw- age, 
grade, sex, and abilit> based on the test publisher's national norms. T^be co nsidered as having 
increased student^rfojm^^ schools should score higher.thaji the if.m^earr predicted score by 
onelSniS^erroro^ or komeothe r band or ra_ngeofscores determined by the district 

to be a realistjc knH appropriate gain in improvement over the schoors predict ed s c o re Schools 
exKbitinglncreased student performance could then be assigned a point value. 

Examples Method 1 

la District with four schools of one type, e.g , high schools: 
Standard Error of £stimate,(SE) =2.5 





Mean 
Predicted 
Score 


One SE or 
District 
IDesired 
Gain 


Criterion 


Actual 
Mean 
Score 


DifTerence 
Between 
Criterion anti 
Actual Score 


School 1 
School 2 
School 3 
School 4 


55 
49 
46 
50 


25 
25 

2 6 
2.5 


57.5 
51.5 
48 5 
52.5 


58 
58 
48 
55 


+ .5 
+ 6.5 
-.5 
+ 2.5 
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CENARIO A(niSTIllCTIM.AN) fi| 



MERITORIOUS SCIIOOlii SCENARIO A(niSTRICTtM.AN) 
(For Illustration purposes only. All items unci point vnlues nre hypotheticnl*) 
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MERITORIOUS SCItOOl^ StJRNARIO B (ItRSULTS FOR SIX SCMOOtnS) 
(Kor Uiuitration purposeionly. All items ancl point vnlucii nre hypothetical.) 
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• 




• 






10 














% 


% 


82 


it 






is 




II 
















S 


n 


9$ 




{horn t«M# 1 brtow) 































a. Tbtrcaretiael^mfnliryffhoottlnlhrelWricl. ... . . , . 

b. Tlitd!9lriclh90#l«lr<8li>eicliti1e9rhievemfnl9Jf>ddMplirw9taiMl«rrf^ . . ^ a^ 
t. ThediBUicI haa e'»ctpd to wf txht r arh caURory difTf r enliy, i.e., verbal and quantiUtivf Iml ac orrji 0 ;I0 pnittt9. pariinpatmn MatularflMO 100 poInU, teichf r atUtii!a7ic<»0 Ml, and home viaiUO 100. 



Student Gain > Verbal and Quantitative t^fU 





Mean 










PrcJirl^ 


Actual 




Point 






TJcore 


pifferentea 






65 


68 


3 


9 


Schttol2 


49 


68 


9 


18 




46 


48 


2 


4 


SrlHM>t4 


M) 


55 


5 


10 


SrhoolS 


4A 


45 


3 


0 




51 


53 


2 


4 



TAHI.K2 

rereentorSttulrntji Who Pars th^ National 1*hyairat KitnrsA T.*tnr% 



PercentPaJiaing 

91100 
flt 90 
71 flO 
CI 70 

5i.r.o 

SOoimI lirlnw 



Polnt.Valur 

50 
40 
25 
10 

5 

0 



TAni.E3 

Number efVoltt^iteer f toura per Student 



Numberj^ifourtt 

0.5 
68 
9 12 
13. 18 
19 and ovar 



PolnlValwe 

IS 
27 
33 
42 

50 



^See eiample lb. psf^e R 4, for eiplanatton of table. 



TABI^4 

Percent ofSludtHttf PaKlcipatInK in School Science Fair 



Pf rcewt of Partici pa nla 

t^i9than25 

2A 35 
3i;'45 
4655 
56 65 
66 and over 



Point Valua 

0 
10 
20 
30 
40 
60 



TAm.K5 
AveraKe Teacher Att^ndanca 
DaysAbtent Point VaJ ue 



0 1 
23 
45 
67 
nior# than 7 



50 
30 
20 
6 
0 



TARUEd 

Percent of Taecharp Mattinx four Home Visitx 



Percont 

80-100 
60 79 
4059 
2039 
1019 

below 10 



Point Value 

100 
75 
50 
25 

5 

0 



Ic. P#reent of st»df »«* »««frding the Predicted fflun Kore. ,C>nce » pr*dicud;sieiin 
score is obuined. the perctritof ttudehtslexce^ 

determined.. EbrrdjitricUs With smrilfjkhwU ©m^^ 'y^'.'-'r'iHS 
schools in the district, those- schools eould-be *»nktd by percent of iludents 
exceeding the predicted nean score is follows: 

School 1 97« 
School 2 
School 3 91% 
School 4 85% 
School 5 67% 
School 6 52% 
School 7 50% 
School 8 40% 
School 9 37% 

The schools could be assigned r^ints by several methods. Following is one 
possible Uble that could be used to determine the achievement poinU for each of 
the nine high schools. 

TABLE B 

Inter>-als , . 

Percent of Students «A*"fr, 
Exceeding Predicted Mean Point \ alue 

95 X 100 

90-94 95 

85 - 89 fg 

80-84 85 



75-79 
70-74 



80 
76 



65-69 II 
60-64 ?9 



55-59 



60 



50-54 55 



etc. 



etc. 



In this example. Schools 1 and 2 will receive 100 points. School 3 will earn 95 
points, and School 4 will receive 90 poinu. etc. These values can then be combined 
with weighted values obtained from the other goal categories included m the 
district's plan to obtain a composite score. The composiU score can then be used to 
determine the meritorious sebools. 



7 



Id. District wiih one school: 

Predicted Score = 52 Standard Error (SE) = 2.5 

Actual Score = 57 



The actual score is more than 2.5 points above ^ « ^^^^ f^S 
school has met the criterion. A point value could be assigned on the basis of a 
SbU dev" o>ed when the district ?l.n is established These pomt ^'^^^^^Jll^ 
be combined with other values obuined from the other eo'»P«n«"" °f 
disScfs plan to develop the composite «ore. An absolute value or cut-off score 
must then be reached for the school to be determined a mentonous school. 



See bold print on page E-l. 
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le. District with three schools, SE = 2.5 



Predicted 
Mean Score 



/ ctual 
S^e&n Score 



Difference 



Schooll 52 55 +3 

School 2 64 67 +3 

Schools 51 63 +2 



Only Schools 1 and 2 have met the critririon because the actual mean for School 3 
did not exceed one SE above th» predicted mean. Only Schools 1 and 2 would 
receive points. These points would be coTobined with points earned in other goal 
categories to produce a composite score for each school. 



Method II 



MuUivariate regression analvsis yielding a mean predicted achi'-ve ment score for a school 
The school should exceed the predicted score by one standard error of tstimate (or some other 
designated band or range of scores esublished in the district plan), be ranked by some method, 
and be assigned a point value which could be combined with other weighted goal category 
values. The regression analysis for each school may include, as independent variables, prior 
achievement, socioeconomic factors such as free or reduced price lunch, racial-ethnic 
characteristics, and/or age. The dependent or criterion variable should be student scale scores 
on any sUndardizcd district-approved achievement test(s). Variations of the analysis based on 
gain scores, grade levels, school characteristics and/or kinds of testCs) may be a part of the 
regression mode: (A discussion and example of a multiple regression cnalysis is in the Student 
Assessment publication. The Third and Fifth Grade Predicted Sc ore Report. 1977>78.) 



Student improvement bv within*school percentile distribution , A within-school percentile 
distribution for each grade tested will be developed from the raw or scale scores of students for 
one year's achievement test and the next year's test, e.g.. 1988 to 1989. Students' raw or scale 
scores on these achievement tests will be converted to a within-school percentile score. 
Students who improve from one year to the next may be counted, e.g., 10 percentile points at 
the middle of the percenUle range (25th percentile to 7£th percentile), 5 percentile points at He 
ends of the percenUle distribution (10th to 25th percentile or 75th percentile to 90th percenwie) 
and 2 percenUle points at Uie very ends of Uie distribution (0 to 10th percenUle and 90th to 
99th percentile). These student counts could then be converted to the percent of students 
improving A school's improvement criterion could be established at a selected percent of the 
students showing improvement by this method. 

Examples- Method Ilia 



Method III 



Points Using Table C 



Schooll 
School 2 
Schools 
SchooU 



75% students improving 
60% students improving 
59% students improving 
58% students improving 



40 

30 
30 
30 



See bold print on page £-1. 
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TABLE C 



Interval 

Percent of Students Assigned 
Improving Point Values 

91 -J^ 60 

81-90 45 

71-80 40 

61-70 35 

51-60 30 

41-50 25 

31 - 40 20 

21-30 15 

11-20 10 

1-10 5 

Using Table C. School 1 earned 40 points znd Sciool 2 earned 30 points. These values ctn be 
combined with other goal categor>- values to form a composite school score and from which the 
meritorious schools can be selected. 

Illb. Assume that 60 percent of the students in a school district of "too few" schools of a 
type improved according to the criteria specified above. 

If the plan set a goal that 50 percent of the students should improve, the goal was 
met A district could esublish a uble to assign point values and combine them 
with results of other goal categories in the district's plan to determine if the school 
met an absolute criterion established in this plan. 

IIIc. District with three schools. 

School 1 = 56^ students improving 
School 2 =^ 52^ students improving 
School 3 = 49<« students improving 

Onlv Schools 1 and 2 met the criterion of 50 percent improving. School 3 did not 
reach the 50th percent criterion. A Uble could be esublished with points for 
intervals of percent of students improving. Assigned points for school 
achievement would then be combined with other district goal values for 
determining the composite score for each school. 



Method IV 



^tnA^nf improvement bv n oti'i^na! p^rcyntilg distribution. This approach is the aame as 
Method in with the exception that the distribution to be used for examining improvement will 
be th(^ national percentile distribution. Therefore, on a nationally-normed test, the studenta 
percentile scores from test forms appropriate for the grade levels could be compared from one 
years performance to another year's performance, e.g., 1988 to 1989. as a measure of 
improvement. 



Method V 



Averaging of previous vears^ nationallv>normed achieveme nt mean scores to determine 
relative measure of improvement . Districts may choose U> average the previous three yearsj 
achievement mean scores and combine the related standard deviations to obtain ail average 
standard deviation. This analvsis could be done by grade. Each grade could then be reqm^^ 
for example, to exceed the previous achievement mean score by one-thu^of the combined 
standard deviation for the school to meet the criterion for receiving poinU. These points would 
be combined with results of other selected district goals to form a composite score for each 



See bold print on page E- 1 . E-7 
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school (Atuchmcnt F conuins the formulas and an example of how to determine the 
combined mean and standard deviations for three years.) 



Examples: Method V 

Va. Assume that School A, Grade 10, had these data: 



Number 



Standard 
Mean Deviation (SD) 



iOth Grade 1985 
10th Grade 1987 
10th Grade 19S8 



250 
225 
275 



SO 
70 
75 



15 
25 
20 



Combined mean ' Tv.l7 (See AtUchment F-i for compuUtion of combined mean 
and sundard deviation.) 
Combined standard deviation - 20.61 
One third ofSD 6.87 

In 1989 the tenth grade test results for School A produced a mean of 79. 

Seventv-nine is not 6.87 (1/3 of combined SD) above the combined mean of 75.17. 
However, if School A had a 1988 mean score of 83, it would have reached the 
criterion. Eighty-three is more than 6.87 points above the combined mean of 
75 17. A Uble of point values could be established and a composite score could be 

determined 

\T>, Use saroe method as above, but average the scores for the same students across 
three years instead of using the same grade three times. 



For this method, a scale score that automatically increases each year would not be 
appropriate. (Many of the test publishers* K.12 scale scores do automatically 
increase each year.) 



Example- 



9th grade 1986 
10th grade 1987 
llth grade 1988 



See bold prim on pag« E-1. 
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APPENDIX G 



SUMMARY OF QUESTIONNAIRE RESPONSES 



m 

-'•Mi 



3?B 



MERIT- SCHOOLS PROGRAM 

Teachers' Responses To Survey 
Salary Supplements vs. Materials Only 



II- 



IS- 

i 



BECAUSE OF THE MERIT SCHOOLS (QUI IP) PROGRAM; 
I. SCHOOL CONDITIONS 



A • agree 

U = undecided 

D = disagree 



^C) 



6" 





%D 


%U 


%A 


Num. 


Student learnina has improved • 










Dade Co 


28 


20 


52 


195 


p<.02 Other Co. w/ Salary Suppo 


28 


35 


37 


94 


Materials only 


20 


30 


* 50 


26.0 


Teachers devote more hours working for the school • 










Dade Co 


18 


15 


67 


197 


p<.02 Other Co. w/ Salary Supp* 


23 


29 


49 


94 


Materials only 


23 


21 


56 


261 


More student time is used for learning activities. 










Dade Co 


29 


24 


47 


196 


n.s. Other Co. w/ Salary Supp* 


30 


• 30 


40 


95 


Materials only 


26 


23 


51 


259 


Teachers use more effective instructional methods. 










Dade Co 


27 


26 


47 


197 


p<.004 Other Co. w/ Salary Supp* 


26 


27 


47 


94 


Materials onlv 


14 


25 


61 


256 


Student attendance is better. 










Dade Co 


28 


21 


51 


195 


p<.001 Other Co. w/ Salary Supp. 


38 


35 


27 


93 


Materials only 


37 


34 


29 


259 


Teamwork among teachers is better. 










Dade Co 


25 


18 


57 


196 


n.s. Other Co. w/ Salary Supp. 


27 


22 


51 


95 


Materials only 


21 


25 


54 


259 



I 

W' ■ 



|) Teamwork among teachers and administrators is better. 



i) 



Dade Co 


24 


21 


55 


195. 






22 


48 


93 


iuauerxaJLo onjiy 


21 


27 


52 


259 


The instructional materials or equipment is better* 










Dade Co 


39 


27 


34 


196 


p<*OUi Utner wo» w/ oaxary aupp* 




28 


37 


95 


nateriaxs oniy 




X u 


79 


261 

^ W JU 


Students take standardized tests more seriously. 








1 * 


Dade Co 


31 


29 


40 


197 




33 


21 


46 


95^: 




35 


36 


29 


260 


More time is used to teach test taking skills. 










p<.02 Dade Co 


17 


17 


66 


197 


otner co • w/ salary bupp • 








94 


Materials only 


22 


23 


55 


258 


Teachers require student participation in activities 










related to merit school goals. 










Dade Co 


12 


13 


75 


196 


p<.04 Other Co. w/ Salary Supp. 


15 


23 


62 


95 


Materials only 


18 


25 


57 


257 



1) Parent participation in school activities is better. 

Dade Co 

n.s. Other Co.- w/ Salary Supp. 

Materials only 



38 


27 


35 


197 


41 


24 


35 


95 


28 


31 


41 


257 



m) Community pride in the school is better. 

Dade Co 

n.s. Other Co. w/ Salary Supp. 

Materials only 



22 


29 


49 


197 


20 


28 


52 


94 


22 


26 


52 


259 



11 
li- 



fe:. 



i 



in 



III. THE MERIT SCHOOLS (QUIIP) PROGRAM: 



A- 4 



A agree 

U = undecided 

D - disagree 



p) 



Id) 









%U 


%A 


Ntan. 


is fair to all schools. 












Dade Co 




38 


24 


38 


409 


p<.003 otner co. w/ salary 


Supp* 


A Q 








Materials only 




25 


34 


41 


511 


has a positive impact on iny school. 












Dade Co 




23 


16 


61 


411 


p<.uU4 otner uo. w/ baxary 


Supp • 


c. D 


1 Q 






Materials only 




10 


18 


72 


521 


provides educators a reward for good work. 












Dade Co 




17 


13 


70 


408 


p<.uui vJuner uo. w/ oaiary 


oupp • 


c, D 


1 n 
± t 




210 


Materials only 




36 


20 


44 


518 


has pushed schooling toward trivial learning. 












Dade Co 




52 


22 


26 


406 


p<.04 Other Co. w/ Salary 


Supp. 


46 


26 


28 


209 


Materials only 




55 


27 


18 


517 


promotes practices that diminish the quality of 
education while test scores increase. 










Dade Co 




56 


20 


24 


405 


p<.001 Other Co. w/ Salary 


Supp. 


AH 




J o 


919 


Materials only 




47 


29 


24 


515 


is very easy, requiring no extra effort. 












Dade Co 




81 


07 


12 


409 


p<,003 Other Co. w/ Salary 


Supp. 


71 


14 




206 


Materials only 




61 


21 


18 


515 


rewards the most improved rather than the highest 
quality schools. 










Dade Co 




23 


32 


45 


405 


p<.01 Other Co. w/ Salary 


Supp. 


19 


25 


56 


212 


jJ^^ Materials only 




20 


37 


43 


514 



}30 



uses test scores that give false information 
t[ about student learning. 



M; Dade Co 



p<.005 Other Co. w/ Salary Supp, 

Materials only 

J/ 

|i) uses other school criteria that are important. 

Dade Co 
Other Co. w/ Salary Supp, 
I n.s. Materials only 



|j) allows politics to get in the way of true teaching 
I quality awards. 

I Dade Co 

n.s. Other Co. w/ Salary Supp. 

I Materials only 

|k) has resulted in wise use of state funds. 

Dade Co 

p<.002 Other Co. w/ Salary Supp. 

Materials only 

: 1) is a credible program for school improvement. 

Dade Co 

p<.02 Other Co. w/ Salary Supp. 

I Materials only 

i 

]o) should be dropped altogether. 

f Dade Co 

I p<.005 Other Co. w/ Salary Supp. 

t Materials only 

) 

|p) should be continued, only if modified. 
f Dade Co 

I p<.002 Other Co. w/ Salary Supp- 

*• 

I: Materials only 



kg) should be continued as is. 

Dade Co 

I p<.001 Other Co. w/ Salary Supp- 

Materials only 

331 



I IV « Background Information 

■a) What term best describes your school: %Elem %HS/JrH 

$ 

Dade Co 62 19 

p<.04 Other Co. w/ Salary Supp. 50 20 

Materials only 57 18 



To your knowledge, for how many years has your school 
participated in the Merit Schools Program? 

%0 %1 %2 %3 %4 %5 



p<.001 Dade Co. 1 1 10 19 25 29 

Other Co. w/ Salary Supp. 1 2 6 31 20 28 
Materials only 1 5 16 36 18 15 

How many times has your school been a Merit (QUIIP) School? 

%0 %1 %2 %3 %4 %5 



p<.001 Dade Co. 14 27 26 18 6 8 

Other Co. w/ Salary Supp. 19 25 28 19 6 3 
Materials only 3 17 19 33 15 10 
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p II > STANDARDIZED TESTS 

IN MiY SCHOOL: 

|i: A = agree 

U = undecided 
D = disagree 

1% %D %U %A 

the curricula are designed to teach the content on 
I*' the test in order to match what is taught and tested. 



m Dade Co 

I' n.s. Other Co, w/ Salary Supp. 

p> Materials only 

I 

|b) teachers are prohibited from reviewing test contents 

I'' Dade Co 

p<.04 Other Co. w/ Salary Supp. 

Materials only 



n 



i 

l^p) items from the tests are used (without modification) 
on other tests or student exercises. 

x Dade Co 

p<,05 Other Co. w/ Salary Supp. 

Materials only 



f^d) items from the tests are slightly modified, then used 
on other tests or student exercises. 

Dade Co 

p<.03 Other Co. w/ Salary Supp. 

Materials only 

e) items on the tests are used as models for item format 
when writing other tests or student exercises. 

Dade Co 

n.s. Other Co. w/ Salary Supp. 

Materials only 

%f) experienced teachers remember items from the test and 
^' focus part of their instruction on those xtems. 

Dade Co 

n.s. Other Co. w/ Salary Supp. 

I Materials only 

I 

fXg) test administration procedures are followed exactly 
fr as printed. 

>^ Dade Co 

I; n.s. Other Co. w/ Salary Supp. 

|: O Materials only 
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40 


21 


39 


209. 


30 


28 


42 


112 


34 


30 


36 


248 


35 


16 


49-, 


210 


32 


14 


54 


113- 


40 


82 


38 


252 


68 


19 


13 


206 


78 


15 


07 


114 


70 


23 


07 


250 


49 


19 


32 


210 


50 


25 


25 


113 


40 


31 


29 


247 


32 


22 


46 


210 


35 


24 


41 


113 


23 


30 


47 


250 


31 


22 


A n 
4 / 


210 


23 


29 


48 


112 


24 


31 


45 


251 


09 


08 


83 


214 


11 


11 


78 


115 


05 


15 


80 


251 



COMPETITION SUPPLEMENT 
TEACHERS' RESPONSES 
BECAUSE OP THE MERIT SCHOOL (QUIIP) PROGRAM: 



if) There is a sense of competition among schools. 

Dade Co. 

p<.002 Other Co. w/ Salary Supp. 

Materials only 

I The school district promotes a sense of 
competition among schools. 

Dade Co. 

p<.03 Other Co* w/ Salary Supp. ^ 

Materials only 

) Students are awaxe of a sense of competition. 

Dade Co. 

p<.01 Other Co. w/ Salary Supp. 

Materials only 

) Teachers are aware of a sense of competition. 

Dade Co. 

p<.001 Other Co. w/ Salary Supp. 

Materials only 

) Merit School competition among schools is good. 

Dade Co. 

n.s. Other Co. w/ Salary Supp. 

Materials only 

) Competition focuses on standardized test scores. 

Dade Co. 

n.s. Other Co. w/ Salary Supp. 

Materials only 

) Competition focuses on other Merit School criteria. 

Dade Co. 

n.s. Other Co. w/ Salary Supp. 

Materials only 

) Competition among schools helps to improve student 
scores on standardized achievement tests. 

Dade Co. 

n.s. Other Co. w/ Salary Supp. 

Materials only 



A = agree 

U undecided 

D ^ disagree 







Tin 




15 


28 


57 




14 


11 


75 


56. 


31 


20 


49 


136 


17 


31 


52 


92?'? 


20 


20 


60 

\* w 


56< V 


33 


23 


44 


138 

xw u 


34 


30 


36 


91 




30 


32 


56 




20 


23 


137 

XW f 


12 


17 


71 


90 


13 


07 


80 


56 

■a' W 


34 


11 
X X 


55 


137 


27 


29 


44 


91 


36 


34 


30 


56 


40 


24 


36 


134 


13 


14 


73 


71 ' 


05 


09 


86 


44 


11 


21 


68 


94 


15 


17 


68 


71 


14 


27 


59 


44 


12 


19 


69 


91 


27 


28 


45 


71 


25 


32 


43 


44 


31 


35 


34 


90 



■A 



f erIc 
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Competition among schools helps to improve student 
academic achievement in general. 

Dade Co, 

p<.02 Other Co. w/ Salary Supp. 

Materials only 

Competition among schools helps to improve student 
achievement of other important school objectives. 

Dade Co. 

n.s. Other Co. w/ Salary Supp. 

Materials only 

Competition among schools helps to improve 
a sense of team cooperation among school 
employees { administrators , teachers , staff ) . 

Dade Co. 

p<.04 Other Co. w/ Salary Supp. 

Materials only 
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MERIT SCHOOLS PROGRAM 
Survey Responses 
P-incipals and Other School Administrators 
Salary Supplements vs. Materials Only 



BECAUSE OF THE MERIT SCHOOLS (QUIIP) PROGRAM: 
I. SCHOOL CONDITIONS 



student learning has improved. 

Dade Co 
Other Co. w/ Salary Supp. 
Materials only 



n.s. 



b) Teachers devote more hours working for the school. 

Dade Co 

p<.0004 Other Co. w/ Salary Supp. 

Materials only 



§) More student time is used for learning activities. 

Dade Co 

n-s- Other Co. w/ Salary Supp. 

i Materials only 

jj) Teachers use more effective instructional methods. 

Dade Co 

p<.02 Other Co. w/ Salary Supp. 

Materials only 

e) Student attendance is better. 

Dade Co 

p<.0001 Other Co. w/ Salary Supp. 

Materials only 

f) Teamwork among teachers is better. 

Dade Co 

p<.001 Other Co. w/ Salary Supp. 

^ Materials only 

^) Teamwork among teachers and administrators is better < 
i- Dade Co 

p<.02 Other Co. w/ Salary Supp. 

^ Materials only 

I ERIC 



A = agree 
U = undecided 
D = disagree 
%D %U %A Num^ 



19 


28. 


53 


72 


36 


32 


32 


50 


19 


27 


54 


94 


21 


24 


55 


72 


54 


14 


32 


50 


38 


31 


31 


94 


26 


25 


49 


72 


46 


29 


25 


48 


28 


31 


41 


93 


20 


35 


45 


71 


41 


22 


37 


49 


19 


.14 


57 


93 


25 


28 


47 


72 


59 


29 


12 


49 


45 


39 


16 




10 


14 


76 


72 


34 


20 


46 


50 


25 


27 


48 


94 


08 


18 


74 


72 


30 


20 


50 


50 


18 


26 


56 


93 



Am 



w 

I- 



The instructional materials or equipment is better. 



Dade Co 


25 


31 


44 


72 


p<.0001 Other Co. w/ Salary Supp. 


50 


18 


32 


50 


Materials only 


08 


08 


86 


91 












Dade Co 


19 


25 


56 


72 


p<.02 Other Co. w/ Salary Supp. 


37 


26 


37 


49 


Materials only 


35 


35 


30 


94 


\Ar\T<^ h n mo iq iiQPf^ to I'oacVi t-pc^* t'alcina skills. 










Dade Co 


11 


17 


72 


71 


p<.003 Other Co. w/ Salary Supp. 


19 


16 


65 


49 


Materials only 


34 


22 


44 


94 


Teachers require student participation in activities 

"roTat-pri t"0 mpTTt' school C103.1s« 










Dade Co 


04 


06 


90 


71 


p<.001 other Co. w/ Salary Supp. 


27 


12 


61 


49 


Materials only 


22 


15 


63 


94 


Parpnt" nar t^iri nat"ion in school activities is better. 










Dade Co 


27 


27 


46 


71 


n.s« Other Co. w/ Salary Supp. 


46 


26 


28 


50 


Materials only 


36 


31 


33 


93 


PoTrmiiin 1 1" V "ott^p in tnp school is bstt&r. 










Dade Co 


14 


25 


61 


72 


p<.02 Other Co. w/ Salary Supp. 


32 


08 


60 


50 


Materials only 


23 


28 


49 


93 


Conmiunication among ad.ministrators from different 

QCHriOl Q T ^ hpt"t"PT 
9^ilWWX.9 Jh^w V«WC?>L * 










Dade Co 


28 


42 


30 


72 


p<.02 Other Co. w/ Salary Supp. 


57 


25 


18 


49 


Materials only 


39 


41 


20 


94 


Teachers tend to transfer from other schools to 
schools that often earn merit school awards. 

Dade Co 


55 


.1 


24 


71 



p<.002 Other Co. w/ Salary Supp. 76 16 08 50 

Materials only 70 26 04 94 



4? 

I 
I?- 



II. TEACHERS ARE MORE INVOLVED IN DECISIONS ABOUT; 



U 
D 



B-3 ' 



agree 

undecided 

disagree 





%D 


%U 


%A 


Num. 


instructional methods 










Dade Co 


11 


24 


65 


72 


p<.02 Other Co. w/ Salary Supp. 


37 


12 


51 


49 


Materials only 


23 


20 


57 


93 


purchase of instructional resources and equipment 










Dade Co 


17 


25 


58 


72 


p<.O001 Other Co. w/ Salary Supp. 


38 


06 


56 


50 


Materials only 


07 


08 


85 


93 


school policy and procedures 










Dade Co 


14 


19 


67 


72 


p<.02 Other Co. w/ Salary Supp. 


40 


18 


42 


50 


Materials only 


23 


21 


56 


94 


school goals 










Dade Co 


10 


11 


79 


70 


p<.03 Other Co. w/ Salary Supp. 


28 


16 


56 


50 


Materials only 


11 


16 


73 


94 


other grade level or school activities and practices. 










Dade Co 


08 


21 


71 


72 


p<.0004 Other Co. w/ Salary Supp. 


39 


16 


45 


49 


Materials only 


16 


28 


56 


94 



I ERIC 



138 



THE. MERIT SCHOOLS (QUIIP) PROGRAM: 



A = agree 

U = undecidad 

D - disagree 













Num 


a) 


is fair to all schools. 












39 


18 


42 


71 




n.s. Other Co. w/ Salary Supp. 


52 


10 


38 


50 




Materials only 


31 


17 


52 


94 




nas a posxTixve xiupaci. on iiiy ouiiuux. 














19 


10 


71 


72 




p<.02 Other Co. w/ Salary Supp. 


32 


14 


54 


50 




Materials only 


11 


09 


80 


93 


c) 


proviues eQucaiiors a xewaxu jlul yuuu wuxjv* 












jk^CtVlC Ww 


17 


10 


73 


73 




n.s. Other Co. w/ Salary Supp. 


32 


18 


50 


50 




Materials only 


18 


15 


67 


93 


d) 


has pushed schooling toward trivial learning. 












Dade Co 


61 


28 


11 


72 




p<.003 Other Co. w/ Salary Supp. 


58 


10 


32 


50 




Materials only 


71 


18 


11 


93 


e) 


promotes practices that diminish the quality of 
ecLucauXOn wniie uesn scores increase. 














63 


22 


15 


72 




p<.002 Other Co. w/ Salary Supp. 


48 


14 


38 


50 




Materials only 


65 


24 


11 


93 


f ) 


is very easy, requiring no extra effort. 












Dade Co 


75 


14 


11 


71 




n.s- Other Co. w/ Salary Supp. 


86 


10 


04 


49 




Materials only 


76 


15 


10 


94 


g) 


rewards the most improved rather than the highest 
quality schools. 












Dade Co 


29 


22 


49 


72 




n.s. Other Co. w/ Salary Supp. 


36 


14 


50 


50 




Q Materials only 


28 


35 


37 


93 



ERIC 



|K) uses test scores that give false information 
|j about student learning. 



Dade Co 


44 


28 


28 


72 


p<.008 Other Co. w/ Salary Supp. 


42 


14 


44 


50 


Materials only 


56 


28 


16 


92 


uses other school criteria that are important. 










Dade Co 


14 


35 


51 


72 


n.s. otner v^o. w/ oaiary bupp. 




OA 






Materials only 




1 / 


CI 




allows politics to get in the way of true teaching 
quality awards. 








• 


Dade Co 


56 


24 


20 


71 


n.s. Other Co. w/ Salary Supp. 


56 


20 


24 


50. 


Materials only 


61 


17 


22 


93 


has resulted in wise use of state funds. 










Dade Co 


15 


33 


52 


72 


p<.OO01 Other Co. w/ Salary Supp. 


46 


28 


26 


50 


Materials only 


14 


31 


55 


93 


is a credible program for school improvement. 










Dade Co 


17 


21 


62 


71 


P<.UUj Uuner uo. w/ oaj-ary oupp. 




^ w 


38 


50 


laateriaxs oniy 


^ V 


1 "X 
X J 


o / 




requires an administrator to devote more time to 
leading school activities. 










Dade Co 


25 


20 


55 


71 


p<.04 Other Co. w/ Salary Supp. 


54 


14 


32 


50 


Materials only 


38 


19 


43 


93 


results in competition that has corrupted schools. 










Dade Co 


69 


16 


15 


71 


n.s. Other Co. w/ Salary Supp. 


67 


12 


21 


49 


Materials only 


11 


17 


06 


93 



B-6 

15) should be dropped altogether. 







Dade Co 


68 


14 


18 


72 


p<.02 




Other Co. w/ Salary Supp. 


43 


31 


26 


49 






Materials only 


72 


14 


14 


93 




















Dade Co 


45 


28 


27 


71 


n • s • 




ui,xiex • w/ oaXaXy oupp* 


J o 


D 


o o 








Materials only 


47 


20 


33 


90 


should be 


continued as is 


• 














Dade Co 


30 


21 


49 


71 


p<.004 




Other Co. w/ Salary Supp. 


62 


12 


26 


50 






Materials only 


33 


17 


50 


93 



1 er|c 



B-7 



r. 



IV, Background Information 

What term best describes your school: 
Dade Co 

I Other Co. w/ Salary Supp. 

I Materials only 

Ic) To your knowledge, for how many years has your school 



Dade Co 
Other Co. w/ Salary Supp. 
Materials only 

d) How many times has your school been a 

Dade Co 
Other Co. w/ Salary Supp. 
Materials only 



%Elein 


%HS/JrH 


%HS 


NUM. 


64 


22 


14 


72 


42 


30 


28 


50 


62 


15 


23 


89 



%0 %1 


%2 %3 4% %5 


%6+ 


Num. 


0 0 


4 15 20 41 


20 


70 


0 4 


2 20 30 30 


14 


50 


1 3 


9 37 25 19 


6 


90 


Merit 


(QUIIP) School? 






%0 %1 


%2 3% %4 %5 


%6+ 


Num. 


14 23 


16 21 7 16 


3 


70 


26 26 


26 10 10 0 


2 


50 


7 8 


13 39 21 9 


3 


90 



ERIC 



U2 



COMPETITION SUPPLEMENT 
SCHOOL ADMINISTRATORS' RESPONSES 



pECAUSE OF THE MERIT SCHOOLS (QUIIP) PROGRAM: 



|a) There is a sense of competition among schools* 

Dade Co. 

p<.0005 Other Co- w/ Salary Supp. 

Materials only 

The school district promotes a sense of 
competition among schools. 

Dade Co. 

p<.0001 Other Co. w/ Salary Supp. 

Materials only 

c) Students are aware of a sense of competition. 

Dade Co. 

p<.001 Other Co. w/ Salary Supp. 

Materials only 

d) Teachers are aware of a sense of competition. 

Dade Co. 

p<,O001 Other Co. w/ Salary Supp. 

Materials only 

e) Merit School competition among schools is good. 

Dade Co. 

p<.008 Other Co. w/ Salary Supp. 

Materials only 

f) Competition focuses on standardized test scores. 

Dade Co. 

n.s. Other Co. w/ Salary Supp. 

Materials only 

g) Competition focuses on other Merit School criteria. 

Dade Co. 

n.s. Other Co. w/ Salary Supp. 

Materials only 

h) Competition among schools helps to improve student 

scores on standardized achievement tests. 

Dade Co. 

p<.02 Other Co, w/ Salary Supp. 

Materials only 

i) Competition among schools helps to improve student 

academic achievement in general. 

Dade Co. 

n.s. Other Co. w/ Salary Supp. 

Materials only 



A = 


agree 






U = 


undecided 




D = 


disagree 




%D 


%U 


%A 


Num. 


17 


18 


65 




26 


06 


68 


47 


48 


11 


41 


88 


15 


22 


63 


68 


38 


09 


53 


47 


50 


18 


32 


88 


28 


32 


40 




38 


26 


36 


47 


54 


21 


15 


88 


13 


16 


71 


o9 


19 


06 


75 


4 / 


42 


18 


40 


88 


16 


25 


59 


69 


36 


28 


36 


47 


43 


30 


27 


87 


16 


12 


72 


b / 


08 


03 


89 


37 


15 


15 


70 


46 


11 


18 


/I 


55 


27 


14 


59 


37 


22 


17 


61 


46 


22 


18 


60 


b/ 


19 


41 


40 


37 


35 


35 


30 


A C 

45 


25 


19 


56 


57 


41 


27 


32 


37 


37 


26 


37 


45 



1^3 



gj?; Competition among schools helps to improve student 
achievement of other important school objectives. 



W. Dade Co. 18 19 63 57 

p<.04 Other Co. w/ Salary Supp. 38 24 38 37 

Materials only 30 33 37 46 



;*:) Competition among schools helps to improve 

p a sense of team cooperation among school 

% employees (administrators, teachers, staff). 

f Dade Co. 19 11 70 57 

I n.s. Other Co. w/ Salary Supp. 27 19 54 37 

Materials only 26 17 57 46 



K o .1 '4 

^ERJC 



m 



mklT SCHOOLS PROGRAM 



c-i 



SUMMARY OF RESPONSES TO SURVEY INSTRUMENTS! 
Teachers vs School Administrators 



BECAUSE OF THE MERIT SCHOOLS (QUIIP) PROGRAM; 
I. SCHOOL CONDITIONS 



Student learning has improved. 



n.s. 



Teachers 
Administrators 



A = 
U = 
D = 

%D 



agree 

undecided 

disagree 

%U %A 



24 
23 



27 
28 



49 
49 



Num. 



549 
216 



b) 



Teachers devote more hours working for the school, 

Teachers 

p<.001 Administrators 



21 
36 



20 
25 



59 
39 



552 
216 



c) 



More student time is used for learning activities. 

Teachers 

n.s. Administrators 



28 
31 



25 
29 



47 
40 



550 
213 



d) Teachers use more effective instructional methods. 

Teachers 

n.s. Administrators 



21 

25 



26 
27 



53 
48 



547 
213 



e) Student attendance is better. 



p<.02 



Teachers 
Administrators 



33 
41 



30 

33 



37 

26 



547 
215 



Jf) Teamwork among teachers is better. 



n.s. 



Teachers 
Administrators 



23 
22 



22 
21 



55 
57 



550 
216 



g) Teamwork among teachers and administrators is better. 

- Teachers 24 24 52 547 

Administrators 18 22 60 215 



n.s. 



I erIc 



j45 



























The instructional materials or equipment is better. 










%3 
>l 


|| Teachers 


25 


22 


53 


552 


3- 

|: n.s. Administrators 


24 


17 


59 


213 


^$ 


if) students take standardized tests more seriously. 












t Teachers 


33 


31 


36 


552 




t n*s. Administrators 


30 


30 


40 


215 




3) More time is used to teach test taking skill'^. 












I Teachers 

I n.s. Administrators 

I 


18 


21 


61 


549 




23 


19 


58 


214 




k) Teachers require student participation in activities 
I related to merit school goals. 










>i 

-Mi 


1 Teachers 


15 


21 


64 


548 




p<.01 Administrators 


17 


11 


72 


214 




1) Parent participation in school activities is better. 












Teachers 


34 


28 


38 


549 




[ n.s. Administrators 


35 


29 


36 


214 




m) Community pride in the school is better. 










1 

"a 


' Teachers 


21 


28 


51 


550 


n.s. Administrators 


22 


22 


56 


215 


' \ 

/r 


n) Communication among administrations from different 
schools is better. 










k 

< 


Administrators 


40 


37 


23 


215 


'i 


o) Teachers tend to transfer from other schools to 










-1 


schools that often earn merit school awards. 












Administrators 


66 


22 


12 


215 


t 

\ 

X 

J 
t 


i 

i, ERIC 5 










1 



r^" ■ ■ 

r 






C-3 






IS- 

1 II. TEACHERS ARE MORE INVOLVED IN DECISIONS ABOUT: 












if 


A = 


agree 






t 




U = 


undecided 






D = 


disagree 






%D 


%U 


%A 


Hum. 




1 

a) instructional methods 












1 Teachers 


25 


20 


55 


552 




t n.s. Administrators 


22 


20 


58 


214 




p) purchase of instructional resources and equipment 










% 


1 Teachers 


20 


16 


65 


551 




1 n.s. Administrators 


18 


13 


69 


215 




p) school policy and procedures 












I Teachers 


31 


25 


44 


552 


-A 


i p< . 0 1 Administrators 


24 


20 


56 


216 


V, 


d) school goals 












t Teachers 


21 


17 


62 


551 




p<.05 Administrators 


14 


15 


71 


214 




5e) other grade level or school activities and practices. 












Teachers 


23 


25 


52 


551 




n.s. Administrators 


19 


23 


58 


215 


% 

t 

-i 


; ERIC J 47 












|gT:"-.„ . ,. _ . . ^ . 
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C-4 



THE MERIT SCHCX3LS (QUI IP) PROGRAM; 

I 
I. 



at) is fair to all schools. 
I p<.001 

I 

b) has a positive impact on my school 
p<.02 



Teachers 
Administrators 

Teachers 
Administrators 



A = agree 

U = undecided 

D = disagree 

%D %U %A Num. 

34 29 37 1126 
39 16 45 215 



17 18 65 1143 
19 10 71 215 



c) provides educators a reward for good work. 

Teachers 

? p<.05 Administrators 



27 
21 



17 
14 



56 
65 



1136 
215 



d) has pushed schooling toward trivial learning. 

Teachers 

p<.004 Administrators 

e) promotes practices that diminish the quality of 
education while test scores increase. 



p<.0Q5 



Teachers 
Administrators 



52 
65 



49 
61 



25 

a.9 



24 
21 



23 
16 



27 
18 



1132 
215 



1132 
215 



f) is very easy, requiring no extra effort. 

Teachers 

■ p<.03 Administrators 

g) rewards the most improved rather than the highest 
quality schools. 

Teachers 

p< . 0 1 Administrators 



70 
77 



21 
30 



14 
14 



33 
26 



16 
09 



46 
44 



1130 
214 



1131 
215 



h) uses test scores that give false information 
about student learning. 

Teachers 



p<.004 



Administrators 



37 
49 



27 
25 



36 
26 



1127 
214 



I er|c 



^ uses other school criteria that are important. 

^ Teachers 

^ p<.002 Administrators 



C-5 



18 
16 



36 
25 



46 
59 



1129 
214 



I*) allows politics to get in the way of true teaching 
rt' quality awards. 



■: p<.001 



Teachers 
Administrators 



38 
58 



28 
20 



34 

22 



1137 
214 



k) has resulted in wise use of state funds. 

Teachers 

'f n.s. Administrators 



27 
22 



31 
31 



42 
47 



1139 
215 



1) is a credible program for school improvement. 

Teachers 

\ n.s. Administrators 



24 
24 



23 
18 



53 
58 



1135 
214 



m) requires an administrator to devote more time to 
leading school activities. 

Administrators 



37 18 45 



214 



n) results in competition that has corrupted schools, 

Teachers 

p<.001 Administrators 



57 
72 



25 
16 



18 
12 



1134 
213 



o) should be dropped altogether. 



p<.03 



Teachers 
Administrators 



61 
64 



20 
18 



19 
18 



1136 
214 



p) should be continued, only if modified. 

Teachers 

p<.001 Administrators 



22 
44 



26 
24 



52 
32 



1119 
211 



q) should be continued as is, 
p<.001 



Teachers 
Administrators 



46 
39 



24 
17 



30 
44 



1126 
214 



lerIc 



t4- 



3^9 



c-6 



a) 



IV. Background Information 

What term best describes your school: 

Teachers 

n.s. Administrators 



%Elem 
58 
58 



%HS/JrH 
18 
21 



To your knowledge, for how many years has your school 
participated in the Merit Schools Program? 

%0 %1 %2 3% %4 %5 
p<.03 Teachers 1 3 12 28 21 23 

Administrators 1 2 62625 28 



%HS NUM. 
24 1110 
21 211 



%6+ Num. 
12 1025 
12 210 



How many times has your school been a Merit (QUIIP) School? 

%0 %1 %2 

Teachers 10 22 23 24 11 08 02 

Administrators 14 17 17 27 14 09 02 



n.s. 



3% %4 %5 %6+ Num. 

1022 
210 



\ ERIC 



150 



W" 


^ 










1 


t 

1 
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1 


i 
c 


COMPETITION SUPPLEMENT 












|bECAUSE of the merit schools (QUIIP) PROGRAM: 

I 

\ 


A = 
U = 
D = 


agree 

undecided 

disagree 




i 

J 
>\ 


I: 

U) 


There is a sense of competition among schools. 

Teachers 

p< . 01 Administrators 


%D 


%U 


%A 


Num. 




22 
32 


13 


D 1 

55 


284 
204 




The school district promotes a sense of 
competition among schools. 

Teachers 

p< . 03 Administrators 


26 
36 


24 
17 


50 
47 


286 
203 




c) 


Students are aware of a sense of competition. 

Teachers 

n . s . Administrators 


46 
Af 


25 
26 


29 
28 


284 
204 






Teachers are aware of a sense of competition. 

Teachers 

n . s . Administrators 


23 
32 


12 
28 


65 
40 


283 
203 




e ) 


Merit School competition among schools is good. 

Teachers 

n . s* Administrators 


35 
32 


28 
28 


37 
40 


281 
203 


< 


f ) 


Competition focuses on standardized test scores. 

Teachers 

n , s . Administrators 


10 
14 


1 ^ 

11 


1 A 

76 


209 
140 


i 


g) 


Competition focuses on other Merit School criteria. 

Teachers 

n.s. Aominxs ura uors 


14 
19 


20 

X o 


66 

O J 


206 
139 




h) 


Competition among schools helps to improve student 
scores on standardized achievement tests. 

Teachers 

n.s. Aanixnxs urd uors 


28 
26 


32 


40 


205 
140 




i) 


Competition among schools helps to improve student 
academic achievement in general. 

Teachers 

n.s. Administrators 


36 
33 


29 
24 


35 
43 


205 
140 




j) 


Competition among schools helps to improve student 
achievement of other important school objectives. 

Teachers 

n.s. Administrators 


32 
27 


24 
z J 


44 

•l o 


207 
140 




k) 


Competition among schools helps to improve 
a sense of team cooperation among school 
employees (administrators, teachers, staff). 

Teachers 

n.s. Administrators 


28 
24 


21 
15 


51 
61 


206 
140 




er|c 



























QU E S T X ONN A X RE 

A = 
U = 
D = 
%D 

!♦ Because of the merit schools program, principals 

and teachers ara interacting to a greater extent 
on a wide range of decisions. 



The merit schools program has added little to 
the activities and behaviors tiiat teachers and 
principals typically perform in our district. 



3. The merit schools program is achieving its primary 
goal of improving instructional practices in our 
district. 



4. Overall, there is a positive feeling among most 
teachers toward the Merit Schools Program. 



5. Because of the pressure caused by the consequences 
of the standardized test scores that are part of 
the ^Serit Schools Program, inappropriate procedures 
or methods are used by teachers and principals. 



6. Our district's merit school program is working 
very well. 



7. District testing experts believe the statistical 
model that interprets the standardized test part 
of the merit schools criteria is a valid model. 

8. There have been few problems with implementing the 
merit schools program. 



9. Transfers by teachers to merit schools have 

increased since the merit school program has begun. 



10. The merit schools program: 

a. should be dropped 

b. should be kept, if modified 

c. should continue in its present form, 



agree 
undecided 
disagree 
%U %A Num. 



42 


15 


42 


26 


54 


15 


31 


26 


31 


23 


46 


26 


19 


15 


46 


26 


77 


08 


15 


26 


15 


23 


62 


26 


23 


19 


58 


26 


27 


04 


69 


26 


84 


10 


06 


25 


45 


10 


45 


20 


14 


36 


50 


14 


33 


11 


56 


18 
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LI. Rank order the benefits of the Merit Schools Program. 



a. The additional dollars it provides to 
teachers and/or schools 



b. The increased interaction between teachers 
and principals 



c. The improvement in standardized test scores 

d. The increased community involvement 

e. The increased school pride 



%1 %2 

65 12 

16 20 

04 12 

04 00 

19 50 



%3 



D-2 



%4 %5 Niiin 



00 12 11 26 

20 24 20 26 

40 36 08 25 

24 28 44 25 

19 04 08 26 
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APPENDIX H 

ANALYSIS OF ATTITUDE, INSTRUCTT, & TEAMWRK INDICES 



TABLE 1 



TEACHER A 
ATTITUDE INDEX 
MEANS 



District 

Dade 

Salary 
Supplements 

Materials 



School Level 
Elementary Middle/ Jr. High High 



32.4 
30.5 

34.6 



36. 1 
27. 3 

33.2 



31.9 
28. 3 

32.3 



Marginals 

32.9 
29.3 

33.8 



Marginals 



33.1 



32.9 



31.4 



Significance of F: 

District .001 
School Level .297 

Interaction Effect: .171 
Pooled Standard Deviation: 9.18 
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TABLB 2 



TEACHER B 
ATTITUDE INDEX 
MEANS 



District 

Dade 

Salary 
Supplements 

Materials 



School Level 
Elementary Middle/ Jr. High High 



33.8 
30.7 

33.7 



33.7 
30.4 

34.1 



35.1 
30.5 

32.5 



Marginals 

34.0 
30.6 

33.5 



Marginals 



33.2 



33.2 



32.7 



Significance of F: 

District .005 
School Level .969 

Interaction Effect: .806 
Pooled Standard Deviation: 9.08 
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TABLE 3 



SCHOOL ADMINISTRATOR 
ATTITUDE INDEX 
MEANS 



District 

Dade 

Salary 
Supplements 

Materials 



School Level 
Elementary Middle/ Jr. High High 



33.7 

29.0 
36.4 



41.4 

30.9 
37.9 



34.4 

36.3 
37.8 



Marginals 

35.5 

31.7 
36.9 



Marginals 



34.0 



37.1 



36.7 



Significance of F: 

District .003 
School Level .030 

Interaction Effect: .180 
Pooled Standard Deviation: 9.26 
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TABLE 4 

TEACHER A 
INSTRUCT INDEX 
MEANS 



School Level 
Elementary Middle/ Jr. High High 



District 

Dade 

Salary 
Supplements 

Materials 



13.1 
12.9 

14.6 



13.5 
12.3 

14.1 



12.4 
11.1 

13.8 



Marginals 

13.0 
12.3 

14.3 



Marginals 



13.7 



13.5 



12.8 



Significance of F: 

District .000 
School Level .072 

Interaction Effect: .769 
Pooled Standard Deviation: 3.60 
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TABLE S 



SCHOOL ADMINISTRATOR 
INSTRUCT INDEX 
MEANS 



District 

Dade 

Salary 
Supplements 

Materials 



School Level 
Elementary Middle/ Jr. High High 



12.9 
11.4 

13. 1 



13.9 
8.8 

15. 1 



11.4 
11.6 

14.1 



Marginals 

13.0 
10.8 

13.6 



Marginals 



12.7 



12.7 



12.7 



Significance of F: 

District . 000 
School Level .769 

Interaction Effect: .076 
Pooled Standard Deviation: 3.84 
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TABLE 6 



TEACHER A 
TEAMWRK INDEX 
MEANS 



District 

Dade 

Salary 
Supplements 

Materials 



School Level 
Elementary Middle/ Jr. High High 



23.3 

23.1 
24.8 



24.8 

20.7 
24.9 



21.9 

19.4 
23.7 



Marginals 

23.3 

21.7 
24.6 



Marginals 



23.9 



24.0 



22.3 



Significance of F: 

District .002 
School Level .069 

Interaction Effect: .418 
Pooled Standard Deviation: 6.54 
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TABLE 7 

SCHOOL ADMINISTRATOR 
TEAMWRK INDEX 
MEANS 



District 

Dade 

Salary 
Supplements 

Materials 



School Level 
Elementary Middle/ Jr. High High 



25.8 
22.2 

24.3 



27.7 
18. 5 

27.5 



23.0 
21.9 

25.2 



Marginals 

26.0 
21.1 

24.9 



Marginals 



24.5 



24.8 



23.6 



Significance of F: 

District .003 
School Level .030 



Interaction Effect: .180 
Pooled Standard Deviation: 6.61 



APPENDIX I 

IMPACT OF SOCIOECONOMIC LEVEL 
AND MINORITY MEMBERSHIP PATTERNS 
OF MERIT SCHOOLS 
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Patterns of Awcirded Schools By County 
Based on Fall 1989 School District Reports 



Very Small and Small County School Districts 



County 



Pattern 



Baker 



Gulf 



1 High School 

1 Middle School 

2 Elementary Schools 



2 Jr/Sn High Schools 

3 Elementary Schools 



a: 
b: 

a: 
b: 
a: 
b: 
c: 



won 2 times 
won 3 times 
won 2 times 
won 3 times 



won 2 of 
won 3 of 
won 3 of 
won 3 of 
won 2 of 



4 times 
4 times 
4 times 
4 times 
4 times 



Madison 



All participating schools won every year, 



Monroe 



^ High Schools 

1 Middle School 

8 Elementary Schools 



a: won 4 of 5 times 
b: won 2 of 5 times mid range SES 
c: won 4 of 5 times 
won 4 of 5 times 
participating elem. schools 
have won 4 of 5 or 5 of 5 
times 



Nassau 



12 schools: most schools have won nearly every time 
they participated, except for two schools. One H.S. 
and one Middle School, both in the mid range SES for the 
district, have won only once. 



Putnajn 



3 High Schools; 



4 Middle Schools: 



two have won 2 of 5 times 
one has won 1 of 5 times (the lower 
SES of the three) 

two have won 2 or 3 of 5 times 
two have won 0 of 5 times (one has 
the lowest SES) 



8 Elementary Schools: one has won 3 of 5 times (lower 

SES) 

two have won 2 of 5 times 
three have won 1 of 5 times 
two have won 0 of 5 times (both 
in the mid-range of SES) 



Patterns of Awarded Schools By County 
Based on Fall 1989 School District Reports 

(continued) 



Very Small and Small County School Districts (continued) 



County 



Sumpter 



Pattern 



2 High Schools: 

2 Middle Schools 

4 Elementary Schools 



both have never won 
one has won 1 time 
three have won 1 time 
One has won 2 times 



Suwannee 



Union 

Wakulla 

Washington 



2 High Schools 

1 Middle School 

2 Elementary Schools 



3 Schools 

5 Schools 

2 High Schools 

2 Middle Schools 

2 Elementary Schools 



one has won 1 time 
one has won 2 times 
has won 2 times 
one has won 2 times 
one has won 3 times 

all three have won at least 1 time 
(information incomplete) 

all have won at least once 

(district participated two times) 



a: won 

b: v;on 

a : won 

b: won 

a: won 

b: won 



of 
of 
of 
of 
of 
of 



5 
5 
5 
5 
5 
5 



times 
times 
times 
times 
times 
times 
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Patterns of Awarded Schools By County 
Based on Fall 1989 School District Reports 



Moderately Small County School Districts 



County Pattern 

3 High Schools each has won 1 of 4 times 

7 Middle Schools four have won 2 of 4 times 

(SES rank: 3, 5, S, 7) {% minority rank: 4, 5, 6, 7) 

one has won 1 of 4 times (upper SES) 
(SES rank : 1) (% minority rank: 3) 

two have not won (mid and upper SES) 
I SES rank : 2, 4) (% minority rank: 1, 2) 

7 Elementary Schools (for which data was available) 

Times won Rank in SES Rank in % minority 

4 (2 Sch) 3, 5 4, 9 

3 (3 Sch) 2, 1, 37 6, 11-5, 15 

2 (8 Sch) 6, 8, 10, 11 1-5, 1.5, 3, 7 

12, 13.5, 15, 16 11,5, 11.5, 16, 17 

1 (2 Sch) 1, 13.5 8, 11.5 

0 {2 Sch) 4, 9 5, 14 



Patterns of Awarded Schools By County 
Based on Fall 1989 School District Reports 

Moderately Small County School Districts 

( continued) 

Lake 7 High Schools 3 have won 1 time 

(SES rank: 1, 5, 7) (% minority rank: 1, 2, 7) 

4 have not won 

(SES rank: 2, 3, 4, 5) (% minor, rank: 3, 4.5, 4.5, 6) 

8 Middle Schools 3 have won 1 time 

(SES rank: 1, 5, 8) (% minority rank: 3, 7, 8) 

5 have not won 

(SES rank: 2, 3, 4, 5, 7) (% minority rank: 1, 2, 4, ^, 6) 

17 Elementary Schools for which data was available 

4 have won 2 times 
9 have ^ on 1 time 
4 have not won 

Times won Rank in SES- Rank in % minority 

2 5.5, 7.5, 7.5, 13 i, 5, 8.5, 16, 

1 2.S, 2.5, 4, 5.5, 10, 2, 3, 6, 7, 11, 
11, 14, 16, 17 12, 14, 15, 17 

0 1, 9, 12, 15 4, 8.5, 10, 13 



Leon 5 High Schools 4 have won 3 times (SES rank: 1, 2, 3, 4) 

(% minority rank: 1, 2, 3, 4) 

1 has won 1 time (SES rank: 5) 
\% minority rank: 5) 

6 Middle Schools 2 have won 3 times (SES rank: 2.5 and 4) 

[% minority rank: 2 and 5.5) 

4 have won 2 times (oES rank: 1, 2.5, 5, 6) 
(% minority rank: 1, 3, 4, 5.5) 

20 Elementary Schools: 13 have won 3 times 

6 have won 2 times 
1 has won 1 time 

Times won Rank in SES Rank in % minority 

3 1, 2.5 4, 5, 6, 7, 8, 9 2, 3, 5, 5, 7, 8, 9, 10 

11, 12.5, 12.5, 14, 17, 11, 12, 13, 14, 16, 

2 2.5, 10, 16, 18, 19, ^0 1, 5, 17, 18, 19, 20 

1 15 15 
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Patterns of Awarded Schools By County 
Based on Fall 1989 School District Reports 

Moderately Small County Sch ^^ l Districts 

(continued) 

Marion 5 High Schools 2 have won 1 time 

(SES rank: 1, 2.5) (% minority rank: 3, 4) 

3 have not won 
(SES rank: 2.5, 4.5, 4.5) {% minority rank: 1, 2. 5) 

6 Middle Schools 4 have won 2 times 

(SES rank: 1, 2, 3, 6) {% minority rank: 1, 3, 4, 6) 
(SES rank: 3) (% minority rank: 3) 

1 has not won 
(SES rank: 1) (% minority rank: 1) 

13 Elementary Schools for which data was available 

6 have won 2 times 
4 have won 1 time 
3 have not won 

Times won Rank in SES Rank in % minority 

2 2, 2, 5, 7, 13, 15 1, 3, 3, 5, 6, 13 

1 2, 5, 9, 12 3, 7, 11, 12 

0 8, 10, 11 8, 9, 10 
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Patterns of Awarded Schools Ey County 
Based on Fall 1989 School District Reports 

Large and Moderate Sized School Districts 

Brevard 10 High Schools 1 has won 4 times 

2 have won 3 times 

1 has won 2 times 
4 have won 1 time 

2 have not won 



Times won 
3-4 

1-2 : 
0 

13 Middle Schools 



Rank in SES 
2, 4-5, 4,5 

. 3, 6, 8, 9,5 

7, 9.5 



Rank in % minority 
1, 2,5, 8 

2,5, 4,5, 4.5, 6, 10 
1, 9 



Times won 
3-4 



2 have won 4 times 

1 has won 3 times 

1 has won 2 times 

4 have won 1 time 

5 have not won 

Rank in SES 

I 9 3 



1-2 6.5, 6.5, 10.5, 12 
0 4, 5, 8, 9, 10.5, 
45 Elementary Schools 



13 



Rank in % minority 
1, 2.5, 4.5 

2.5, 7, 11, 12, 13 

4.5, 6, 8, 9, 10 



Times won 

4-5 
3 

1 

0 



3 have won 4 or 5 times 

7 have won 3 times 

9 have won 2 times 

9 have won 1 time 

17 have not won 



Number of Schools in Each Group 
% Free or Reduced Lunch % Minority 

0-19 20- 39 40 + 0-9 10-19 



20+ 



1 
4 

4 

4 



4. 



3 
3 



3 
4 
5 
3 
8 



2 
3 
6 
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vdrge and Moderate Sized School Districts 



(continued) 



Dade 



24 High Schools 



6 have won 3,4, or 5 times 

I has won 2 times 

4 have won 1 time 

13 have not won 



N^unber in 

Times won Free or Reduced Lunch 

0-9% 10-19% 20% + 

0 2 7 4 



1 or 2 1 
3, 4, or 5 4 



0 
0 



Number in 
% minority 
0- 74% 75-89% 
3 3 



2 
4 



2 
1 



90% + 
7 

1 
1 



47 Middle Schools 



8 have won 3,4, or 5 times 
22 have won 1 or 2 times 
17 have not won 



Times won 



Number in 
Free or Reduced Lunch 



Number in 
% minority 



ERIC 





0-34% 


35 


-54% 


55% + 


0- 74% 


75-89% 


90% 


0 


-) 


7 




8 




5 


5 


7 


-r 2 


10 


6 




6 




9 


6 


7 


4, or 


5 2 






4 




0 


'^ 
C 


o 


179 


Elementary Schools 21 

30 
51 
77 


have won 
have won 
have won 
have not 


3, 4, or 5 
2 times 
1 time 
won 


times 




Times 


Number in 
won % Free or Reduced 
0-39% 40-59% 


Lunch 
60+ % 


0-74% 


Number in 
% Minority 
75-89% 90% + 


3/4, 


or 5 10 

(19%) 




2 

( 5%) 




9 

(10%) 


10 
(17%) 


4 

(10%) 


7 

( 9%) 


2 


4 

( 8%) 




5 

(13%) 




21 

(24%) 


5 

• ( 9%) 


4 

(10%) 


21 
(26%) 


1 


12 
(23%) 




13 
( 33%) 




26 
(30%) 


16 
(28%) 


9 

(23%) 


26 
(32%) 


0 


26 

(50%) 




20 
(50%) 




31 
(36%) 


(47%) 


23 
(58%) 


27 
(33%) 




(100%) 




( 100%) 




(100%) 


(100%) 


(100%) 


(100%) 
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Patterns of Awarded Schools By County 
Based on Fall 1989 School' District Reports 

Large and Moderate Sized School Districts 
{ continued) 

Orange 10 High Schools 1 has won 2 times 

6 have won 1 tiine 
3 have not won 
Times won Rank in SES Rank in % minority 

2 6.5 4 

1 1, 3*5, 3.5, 3.5, 6.5 8 1, 2, 3, A, 9, 10 

0 3-5, 9,5, 9.5 5, 7, 8 

16, Middle Schools 14 have won 2 times 

2 have won 1 time 

SES rank of 1 time winners: 13, 14 
% minority rank: 14, 15 

72 Elementary Schools 53 have won 2 times 

12 have won 1 time 
7 have not won 

Number in Number in 

Times won % Free or Reduced Lunch % Minority 

0-30% 31-50% 51+ % 1-25% 26-50% 51% + 



2 


21 


16 


16 


20 


19 


14 


1 


1 


4 






3 


7 


0 


4 


2 


1 


0 


2 


2 
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Laiqe and Moderate Sized School Districts 
(continued) 


''A 


Pasco 


6 High Schools 1 has won 4 times 

2 have won 3 times 

3 have won 2 times 


• 




Times won Rank in SES Rank in % 
4 5 6 


minority ^ 




3 2, 4 2, 5 






2 1. 3.5, 3.5 1, 3, 4 


\ 




7 Middle Schools 2 have won 3 times 

0 has won 2 times 
3 has won 1 times 
2 have not won 






Times won Rank in SES Rank in % 
3 1, 4 1, 7 


minority 




2 






1 2, 4, 6 2.5, 5, 


7 




0 4, 7 2, 6 






23 Elementary Schools 8 have won 2 times 

7 have won 1 time 

8 have not wen 






Times won SES Rank % 

0-9% 

2 3, 5, 7.5, 10, 11, 13, 17, 20 4 


Minority Gp. 
10-19% 20% + 
2 2 




1 1, 2, 6, 9, 13, 13, 21 6 


0 1 




0 ^, 7.5, 15.5, 15.5, 18, 19, 22, 23 7 


1 
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Large and Moderate Sized School Districts 
{ continued) 

Seminole 6 High Schools 3 have won 1 time 

3 have not won 

Times won Rank in SES Rank in % minority 

1 2.5, 4, 6 2, 3, 6 

0 1, 2.5, 5 1, 4, 5 



9 Middle Schools 4 have won 2 times 

4 have won 1 time 
1 has not won 



Times won Rank in SES 

2 1, 4.5, 7, 9 

1 2, 3, 4.5, 6 

0 8 



Rank in % minority 
3, 6, 8, 9 

i, 2, 4, 5, 

7 



27 Elementary Schools 8 have won 2 times 

10 have won 1 time 
9 have not won 

Number in Number in 

Times won % Free or Reduced Lunch % Minority 

0-15% 16-30% 31+ % 1-20% 21-40% 41% + 

2 2 3 3 3 4 1 

1 5 3 2 7 1 2 

0 2 5 2 4 4 1 
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Large and Moderate Sized School Districts 



(continued) 



Volusia 6 High Schools 



Times won 

3 

2 
1 

R Middle Schools 



1 has won 3 times 
1 has won 2 times 
4 have won 1 time 



Rank in SES 
1.5 



Rank in % minority 
3 



Times won 
3 

2 

1 

0 



1.5, 3,5, 3.5, 6 

1 has won 3 times 
4 have won 2 times 

2 has won 1 times 
1 has not won 

Rank in SES 
5 

1.5, 1,5, 4, 7.5 
3, 7.5 
G 



1, 2, 4, 5 



Rank in % minority 
1 

2.5, 2.5, 5, 7 

4, 8 
6 



35 Elementary Schools 



3 0 have won 3 times 

12 have won 2 time 

4 have won 1 time 

9 have not won 



Number in 

Times won % Free or Reduced Lunch 
0-29% 30-59% 60+ % 



Number in 
% Minority 
1-15% 16-30% 31% + 



